Orthogonalization by Principal Components Analysis

Principal components analysis is a prime tool for obtaining sets of orthogonal variables. In this chapter we discuss how to obtain a set of orthogonal variables from the principal components solution and what these sets are useful for. To obtain the orthogonal variables by using principal components analysis we have to compute factor scores.

To compute the factor scores, one has first to obtain the matrix of eigenvalues from the matrix of factor loadings, as

 

 

The matrix L contains the eigenvalues in its principal diagonal and zeroes in its off-diagonal elements. Thus, for the example from the previous chapter,

 

 

 

The matrix equation for computation of the matrix of factor score coefficients B is

 

 

 

and can be obtained by pre-multiplying the inverse of the diagonal matrix of eigenvalues L by the matrix of factor loadings P. For the example,

 

 

 

it can be computed as follows. First, compute the determinant of the matrix L as (1.30)(.70) - (.00)(.00). The determinant is non-zero (.91); thus the matrix L is invertible. Next, invert the matrix L as

 

 

 

Finally, post-multiply the matrix P, by the inverted matrix L as

 

 

 

Orthohogonalization of Variables

The matrix of the factor scores, B, can be used to orthogonalize the data matrix, incipient to the analysis, as

 

 

For the example, the data matrix can be orthogonalized as

 

 

 

Prior to principal components analysis we had two variables, and  which correlated at .30. After the analysis we obtained two principal components, mutually orthogonal.

Correlations between Principal Components and the Criterion Variable

Consider the previous example of multiple regression, however this time with an orthogonalized set of predictor variables  and  

 

 

The matrix of coefficients of determination between variables ,  and Y is

 

 

 

In this special case, and the squared multiple correlation can be computed directly as

 

 

 

which agrees with the  value obtained earlier.

Orthogonal Codes Obtained by Principal Components Analysis

Since the principal components solution is primarily structural we can use linear transformations to change it to some other form. Within the framework of the analysis of variance designs, a set of orthogonal coding vectors, indicating group membership, is frequently required. It is often, but not always, possible to transformation principal components into integer numbers by finding, in each column, the absolute value of the smallest element (except zero) and divide all elements in this column by this value. For our example, the matrix of normalized eigenvectors

 

 

 

was transformed to integer numbers by dividing  by ,44 and  by .60 as

 

 

 

The matrix of coefficients of determination for the above variables is

 

 

 

the same as the matrix obtained for the normalized eigenvectors.