Interpretation of Factor Analysis

The n-dimensional space can be described by a system of n coordinates. The orientation of coordinates is often arbitrary and the coordinates can be rotated, as to optimize some property of the measured system. There is an infinite number of possible orientations of a coordinate system so selection of a particular orientation depends to a degree on the researcher's perception of properties of the solution to be stressed. The arbitrariness of a system of coordinates can be well illustrated by considering the geographical system of latitudes and longitudes. The passage of the zero meridian through Greenwich is optimal only to the English navigators.

          In factor analysis, reference axes are rotated to increase interpretability of factors. Factor loadings can be rotated; i.e., described by a different system of coordinates, either visually or analytically. Depending on angular separation of the reference axes, the rotation can be either orthogonal or oblique. The best orthogonal analytic rotation method is Kaiser's Varimax. Other orthogonal rotation methods are quartimax and equimax. Oblique analytical methods of rotation show far greater variety of methods than the orthogonal rotation methods.  Among the better known techniques of oblique rotations are promax, maxplan and direct and indirect oblimin.

Transformation Matrix

To rotate a system of Cartesian coordinates counterclockwise around the origin, use a transformation matrix

 

 

 

Consider a point with Cartesian coordinates (1,1), as shown below, to be rotated 45 degrees, counterclockwise.

 

 Both the cosine and sine of 45 degrees are .71, thus

 

 

 

results in the vector of the new coordinates of the point (0,1.4). The Pythagorean theorem can easily verify the new coordinates. The hypotenuse of the triangle connecting the original point with the origin of the Cartesian coordinates equals ; i.e., ; which is 1.4.

Trans-temporal Cognitive Matching

It was suggested that the Zeitgeist period of the 1960's and 1970's paralleled the period of European romanticism around the turn of the 18th and 19th centuries. Hypotheses of this type are frequently made within the non-quantitative areas of the social sciences as history or philosophy. The following experimental investigation attempted to quantify this hypothesis.

          To measure a societal climate, sentences from popular books, expressing opinions, were converted into agree-disagree test items and administered to a group of 63 subjects. The societal climate of the classical romantic period was assessed by Goethe's book Die Leiden Des Jungen Werthers (Goethe). The contemporary social climate was represented by two bestsellers; by Reich's book The Greening of America and by Piersig's book Zen and the Art of Motorcycle Maintenance (Zen). Sentences from Reich's book were used to construct the neo-romantic Consciousness III scale (CIII) and Consciousness I and II scales (CI and CII), indexing the non-romantic, analytical and rational set of attitudes. Additional scales were added to the analysis, indexing the normative attitudes (Norm), absorption - openness to self-absorbing and self-altering experiences (Abs) and measuring cognitive styles characteristics of the Euro-American (West) and Oriental (East) Civilizations. The scores on all nine scales analyzed were correlated using product-moment coefficients of correlations, as shown in the following table.

 

 

 

The above correlation matrix was factor-analyzed. Results of this analysis are described in the following paragraphs.

          The first statistics of interest from the generated output is the determinant of the correlation matrix. For the example, it was computed as .03 and used for computation of the Bartlett's sphericity test

 

 

 

where n equals the number of subjects and k is the number of variables. The Bartlett's sphericity test tests the null hypothesis that the population correlation matrix is an identity matrix. If the obtained chi square value is significant, then the correlation matrix to be analyzed is non-random. The better procedure is to use Humphrey and Ilgen's parallel test.

Communalities, Eigenvalues, and Eigenvectors

The squared multiple correlations, obtained from the inverted matrix of inter-correlations were used as communality estimates. The communalities and eigenvalues are shown in the following table.

 

 

 

Eigenvalues are plotted below where Cattell's scree can be observed.

 

Both the scree of eigenvalues and Kaiser's rule indicated that two factors should be extracted. Following iterations, a matrix of factor loadings was obtained as

 

 

 

 

Varimax Rotation

For many years, Thurstone's criteria of simple structure were used for graphic rotations. These criteria consisted of rules describing favorable properties of the rotated solution as, e.g., that each row of the matrix of factor loadings should have at least one zero and that each factor should have a distinct set of factor loadings close to zero. However, the graphic rotations were partly subjective and did not lead to unequivocal solutions. Analytic rotations approximate Thurstone's criteria better. Among them, Kaiser's Varimax is the most widely accepted method for analytical rotation. This method uses the iterative maximization of column variances of the factor loadings. For the example, the Varimax rotated factor loadings were

 

 

 

Rotated factor loadings are used for naming of the obtained factors. The highest factor loading in each row is highlighted. You look down each column and ask what have the variables with highlighted factor loadings in common. For our example, we named the first factor East  reflecting the cognitive styles of the East civilizations. The second factor we named West - reflecting the cognitive styles of the West civilizations. You may also rearrange the matrix of factor loadings accordingly

 

 

 

For the example, the transformation matrix, obtained form the application of Kaiser's Varimax was

 

 

Since  equals about 5 degrees, as well as , the adjustment of factor loadings for our example will be only minor. The Varimax rotated factor loadings are shown in the figure below.

The East, Goethe, Zen, Consciousness III and Absorption scales are located in the top cluster. The Consciousness II, Consciousness I, Normative, and West scales are clustered toward the right of the above figure. During the rotation, communalities of the rotated and the unrotated solutions remain identical. However, the variance of the columns of the factor matrix increases. The Varimax rotation facilitates interpretation of factors by increasing their variance and thus their information content.