Interpretation of Canonical Analysis

The intra-variate structure can be interpreted as describing correlations between a variable and a canonical variate, which is analogous to a definition of factor loadings, defined in factor analysis as the correlation between a variable and a factor. However, interpretation of the results of canonical analyses that have not been rotated often does not make sense.

Rotation in Canonical Analysis

As in factor analysis, rotation of intra-variate structure, or indirectly, rotation of canonical weights, can improve interpretability of the canonical solution.  For the example, the Varimax rotated canonical weights were obtained from the computer-assisted solution of Kaiser's algorithm and are presented in the following table.

 

 

 

 

The rotated intra-variate structure was computed by using equations

 

and

 

For the example, these operations can be illustrated as

 

and

 

 

These computations are summarized and annotated in the following table.

 

 

 

 

Canonical Correlations for the Rotated Structure

Following rotation, canonical correlations have to be recomputed, as rotation changes the locations of both the left and the right canonical variates within the hyperspace. The matrix of rotated canonical correlations can be obtained from the rotated canonical weights as

 

 

For the example

 

and

 

         

Let us compare this matrix of rotated canonical correlations with a matrix of canonical correlations obtained from the initial solution as

 

 

 

First, compare the traces of both matrices (1.21) and their squares (.76). The traces of both rotated and unrotated matrices of canonical correlations and eigenvalues should remain identical. This equivalence suggests that the amount of predictable variance was not affected by the rotation. Next, consider the off-diagonal elements, indicating orthogonality of the unrotated solution and obliqueness of the rotated one. Finally, consider the magnitudes of values in the main diagonal of both rotated and unrotated matrices. The rotated canonical correlations are distributed more evenly among the canonical variates. This is analogous to changed distribution of factor variance contributions in factor analysis, following Varimax rotation. Since the maximally interpretable factors (canonical variates) are those conveying the most       information, this redistribution of variance indicates that the rotated structure should be more interpretable.

K-Fold Cross-validation of Canonical Analysis

Canonical correlation is Pearson's product-moment correlation between two linear functions that were extracted from two sets of variables. As in the case of the multiple R, the canonical correlations are sensitive to sample-specific covariation. To remove the sample specific covariation, the canonical analysis should be cross-validated. To multiply cross-validate the canonical analysis, the data set is randomly split into two sub-samples of about equal size. For each sub-sample, canonical weights and canonical correlations are extracted. The canonical correlations are normalized and fitted to a running composite. The iterations are continued, until the composite stabilizes. At the termination of iterations, the running composite of the cross-validated canonical correlations is de-normalized, producing a k-fold cross-validated canonical correlation.

Neoromantic America Revisited

In a study pertaining to the dimensionality of the East-West scales, it was hypothesized that Consciousness III as contrasted with Consciousness I and II sets of values are related to the right-left polarity of political thought. Scales utilized in this analysis were similar to scales discussed in previously described trans-temporal cognitive matching experiment.

          The criterion set consisted of the Consciousness scales, derived from Reich's book The Greening of America. The set of predictor scales was comprised of Normative-Humanistic scales, measuring the right-left polarity of values. These scales were developed within the context of Tomkin's structure of ideological systems. Additional predictor variables were the Authoritarian scale and the Absorption scales. The Absorption scale measured openness to self-altering experiences. It could have been conjectured a priori that the Normative and Authoritarian scales would be associated, as would be the Humanistic and Absorption scales. The correlations between the Consciousness scales, ideological polarity scales, and age of subjects were computed as presented in the following table.

                                                         

 

 

 

and analyzed by a program for canonical analysis. The obtained set of canonical weights is presented in the following table.

 

 

 

 

 

and plotted, as

The matrix of canonical weights was rotated analytically by Kaiser's Varimax an plotted, as

 

 

The rotation was via the transformation matrix

 

 

 

 

For the example, the values of sine and cosine elements of the above matrix were

 

 

 

 

The  as well as well approximated the rotation angle that could have been selected by the visual rotation. The resulting rotated matrix of canonical weights is presented in the following table.

 

 

 

This rotation is interesting, since it was the first rotation of canonical weights. The rotated canonical weights can be easily interpreted as showing substantial congruence with the structure of data that could have been expected prior to analysis.

Contrasting Rotated and Unrotated Solutions

In the middle seventies, the skepticism about interpretability of canonical analysis was widespread. Paul Warwick warned that there can be no assurance that canonical analysis will find any comprehensible patterning to the data at all. He also observed that canonical analysis can produce canonical variates, even clearly defined ones, that may not make any sense to the researcher. In a similar vein, Fred Kerlinger stressed that considerable caution and circumspection should be exercised in the interpretation of the canonical regression weights.

          Following the encouraging visual rotation of canonical weights for the Consciousness III: Fact or Fiction? study and theoretical conceptualization of the rotation problem, we were looking for confirmation of the soundness of the idea of rotating the canonical regression weights to improve the interpretability of canonical intra-variate structures. This opportunity presented itself when reading the newly published Primer of multivariate statistics by Richard Harris. Illustrating the section on canonical analysis was a study by Ferraro and Billings on the drug use at the University of New Mexico. Based on the responses of 2,675 undergraduates, a canonical analysis was used to show relationships between a set of variables pertaining to socioeconomic status, gender, age, religion, politics, attitudes toward war in Vietnam, abortion, etc., and a number of questions pertaining to the use of marihuana and opinions about its legalization.

          Reading the section on interpretation of the obtained canonical weights, we came across this interpretation of the third canonical variate. 'Young females who tend not to have used marijuana in the past nor to be using it at present, but nevertheless are more in favor of legalizing marijuana for everyone, 18 years old, or not.' Occam's Entia non sunt multiplicanda praeter necessitatem embodied in the "but nevertheless." Richard Harris graciously provided the original correlation matrix and, following the rotation, it turned out that it is teenage males who tend to advocate the age-unrestricted legalization of marihuana. Why should mothers push drugs on their children?