Eigenanalysis is a prime tool for obtaining sets of orthogonal variables. In this chapter we discuss how to obtain a set of orthogonal variables from the principal components solution and what these sets are useful for. To obtain the orthogonal variables by using principal components analysis we have to compute factor scores.
To compute the factor scores, one has first to obtain the matrix of eigenvalues from the matrix of factor loadings, as
The matrix L contains the eigenvalues in its principal diagonal and zeroes in its off-diagonal elements. Thus, for the example introduced in the previous chapter,
The matrix equation for computation of the matrix of factor score coefficients B is
and can be obtained by pre-multiplying the inverse of the diagonal matrix of eigenvalues L by the matrix of factor loadings P. For the example,
it can be computed as follows. First, compute the determinant of the matrix L as (1.30)(.70) - (.00)(.00). The determinant is non-zero (.91); thus the matrix L is invertible. Next, invert the matrix L as
Finally, post-multiply the matrix P, by the inverted matrix L as
This operation results in the matrix of factor score coefficients, B. The matrix B can be annotated as
The matrix of the factor scores, B, can be used to orthogonalize the data matrix, incipient to the analysis, as
For the example, the data matrix can be orthogonalized as
The properties of the original data matrix and that of the orthogonalized data matrix can be summarized, as shown in the following table.
The correlation coefficient between
original variables
and
was computed in column seven as equal to
.30. The correlation between linearly transformed variables
and
was computed in column ten as equal to
zero, indicating that these transformed scores are
orthogonal, i.e., not correlated.
Prior to principal components analysis we had two variables, Z1and Z2 which correlated at .30. The scatterplot can be shown blow.
After the analysis we obtained two principal components, mutually orthogonal. The scatterplot can be shown blow.
Compare these two scatterplots and you can see that the variance along the first principal axis of the bottom picture is maximized. The first principal component extracts as much variance from the set of variables as possible.
These findings may be conveniently summarized as shown in the following table.
Of interest also is the matrix of squared coefficients, presented in the following table.
where the proportions of variance accounted for are summarized.
Consider the previous example of
multiple regression, however this time with an
orthogonalized set of predictor variables
and
as presented in the following table.
The correlation matrix between
variables
,
and Y is presented in the following table.
In this special case, the difference between
cross-correlations and beta weights disappears, as in the
case of bivariate prediction when the beta weight is equal
to the coefficient of correlation.
Since the inverse of an identity matrix is again an identity matrix, the vector of beta weights is equivalent to the vector of cross-correlations [.75 -.17] and the squared multiple correlation can be computed directly as a sum of squared beta weights, i.e.
and
which agrees with the
value obtained earlier.
Since the principal components solution is primarily structural we can use linear transformations to change it to some other form. Within the framework of the analysis of variance designs, a set of orthogonal coding vectors, indicating group membership, is frequently required.
It is often, but not always, possible to transformation principal components into integer numbers by finding, in each column, the absolute value of the smallest element (except zero) and divide all elements in this column by this value. For our example, the matrix of principal components
was thus transformed to integer numbers as
The correlation matrix for the above variables is shown in the following table.
This matrix is the same as the correlation matrix obtained for the principal components.
In the previous chapters we illustrated the multiple regression analysis by analyzing incarceration rates for the countries listed in the table below, as related to the unequal distribution of wealth and divorce rates.
Using the principal components analysis to orthogonalize the set of predictor variables, the predictor variables in the above table can be replaced by their principal components
The matrix of coefficients of correlations, obtained from the above table is
As the predictor variables are orthogonal, the coefficient of multiple determination can be obtained by summing the squared correlations between the predictor variable and the criterion as .87 + .01, equal to .88. The distribution of component variances of the coefficient of multiple determination is markedly uneven, as the first principal component extracts as much variance from the set of predictor variables as possible.
