Factor Analysis

Let us consider an extended definition of the specification equation of the variance components of the total variance. So far we have discussed only the specification equation for the multiple regression, partitioning the total variance in the criterion variable into the information and error components. Sometimes it is feasible to conceptualize the specification equation as that the total variance consists of three components: common variance, variance specific to a variable and error variance, i.e.,

 

 

 

 In the above equation the common variance was designated as , the specific variance as and the error variance as .

The Number of Factors to Extract

Several criteria were proposed in this respect. Perhaps the best is the Humphrey-Ilgen Parallel Analysis. To use this procedure, two data matrices are factor analyzed simultaneously and their eigenvalues are plotted. One is the analysis of the 'real' data. The second is the analysis of random numbers. The matrix of random numbers should have the same size as the matrix of "real" data. The intersection point of eigenvalues from both solutions determines the number of factors to be extracted. Another criterion is Cattell's Scree Test. Eigenvalues, this time obtained from a single analysis, are plotted and an inflection point of the resulting curve (scree) is determined by visual inspection. The location of the inflection points indicates the number of factors to be extracted. Graphical plots for these criteria can be found on the multimedia compact disk, complementing this book.

          The third procedure often used in this context is Kaiser's Criterion, stating that, as many factors should be extracted as variables with eigenvalues greater than or equal to one. The rationale behind this criterion is that interpretation of proportions of variance, smaller than the variance contribution of a single variable, are of dubious value. Kaiser's criterion is the one most frequently used since it does not require visual inspection of eigenvalue plots and is easily computerized. For our example, there was one eigenvalue close to one , and we decided to extract one factor.

Eigenvalues

The equation to be solved to determine eigenvalues of the principal factor solution proper is

 

 

 

where C is a matrix with communalities in its principal diagonal and correlations in its off-diagonal elements. For the example

 

 

Computing the determinant

 

 

results in a quadratic equation

 

 

Solving by quadratic formula, for the first root only (only one factor has to be extracted),

 

so

 

and

 

Division of the eigenvalue by the trace of the original correlation matrix R (for our example the trace equals 2) gives the proportion of the variance extracted as .96/2 = .48. For two variables, this can be written as

 

 

Solving the determinant

 

 

and expanding the binomial

 

 

gives the first root of the quadratic equation as equal to

 

so

 

For the example, the first eigenvalue equals .36 + .60 which equals .96

 

Eigenvectors

The computation of eigenvectors follows the same procedure as described earlier for the principal components solution. The characteristic equation is solved, leading to the structural, indeterminate solution

 

 

The structural solution thus can be conceptualized as

 

 

Imposing the restriction that the column sum of the squared factor loadings has to equal the first eigenvalue

 

 

 

By taking square roots of the above eigenvector results in a matrix (in this case a vector) of factor loadings

 

 

 

As pointed out earlier, a factor loading is a correlation of a variable with a factor. The matrix of factor loadings thus permits interpretation of a factor in terms of common properties of variables.