Canonical Analysis

Canonical analysis is an extension of multiple regression analysis from one criterion variable to a set of criterion variables. Using multiple regression, we analyzed relationship of a set of predictors to a single criterion variable

 

 

with the correlation supermatrix corresponding to the above data matrix

 

 

I canonical analysis we analyze relationship of a set of predictor variables to a set of criterion variables

 

 

With the correlation supermatrix corresponding to the above data matrix

 

 

The above supermatrix consists of four submatrices: the submatrix of inter-correlations between predictor variables , the submatrices of the cross-correlations between predictor and criterion variables and its transpose, and the submatrix of inter-correlations between criterion variables .

Fundamental Equation of Canonical Analysis

The fundamental equation of the multiple regression analysis is

 

 

The fundamental equation of the canonical analysis is

 

 

While in multiple regression analysis the expression on the left side of its fundamental equation is a scalar number, the coefficient of multiple determination, R², the expression on the left side of the fundamental equation of canonical analysis is an asymmetric matrix M.  As the extant algorithms for eigenanalysis can extract eigenvalues only from symmetric matrices, the canonical matrix M is typically defined as

 

 

 

where A, a symmetric matrix, equals

 

 

 

and B, also a symmetric matrix, equals

 

 

 

A typical algorithm for eigenanalysis of asymmetric canonical matrices extracts eigenvalues from both A and B matrices separately, inverts the A matrix, and recombines the extracted eigenvalues. Next, it extracts and normalizes the right eigenvectors, D, i.e., the eigenvectors corresponding to the criterion set of variables. Imposing restrictions on the structural solution as to conform to the equation

 

 

 

normalizes the right eigenvectors D. The left eigenvectors, C, are then obtained from the equation

 

 

 

The matrices of standard canonical weights C and D are analogous to a vector of standard partial regression coefficients beta or to the matrix of factor score coefficients, B.  Thus the notation designating the right and left sets of standard canonical weights as C and D observes the alphabetic progression B, C, and D.

          As in the principal components analysis where the expression

 

 

 

makes the data vectors orthogonal, the definitions of the canonical weights within the canonical analysis as

 

 

and

 

 

 results in two sets of mutually orthogonal canonical variates U and V.

 

          At this point, let us introduce a numerical example based on a hypothetical data matrix presented in the following table.

 

 

As a first step, let us standardize the above matrix and refer all subsequent discussion to this standardized data matrix. To simplify the notation, instead of using the "z form" for the standard scores, let us use the "obtained scores" form when referring to standard scores. The  thus becomes X,  becomes Y, etc., as shown in the following table.

 

 

The matrix of Pearson's product-moment correlations corresponding to the above data is

 

The canonical analysis commences by computing A and B components of the canonical matrix M. For the example, the matrix A equals

 

 

and the matrix B equals

 

 

 

The matrix M can be computed as

 

 

which equals

 

 

Eigenvalues and Canonical Correlations

For the example, the matrix M is markedly asymmetric. There is another canonical matrix, N, defined as

 

 

 

Although matrices M and N are different, their eigenvalues are identical. For our example the matrix N happens to be close to symmetric, as

 

 

 

Thus, the eigenvalues of the matrix N can be approximated as

 

 

 

The above equation, for the example, equals

 

 

Computation of the determinant results in quadratic equation

 

 

which can be solved by the quadratic formula as

 

where

and

 

The canonical correlations  and  are just the square roots of their corresponding eigenvalues  and . Thus, for the example, the first canonical correlation  equals .99 and the second canonical correlation  equals .22. The correct values of the eigenvalues obtained from the asymmetric eigenanalysis are .99016 and .04734 with the corresponding canonical correlations equal to .995 and .218.

          Summarized, the results obtained so far can be presented as in a typical output from a program for canonical analysis, are presented in the following table.

 

 

The canonical correlations in the above table are analogous to multiple R of the regression analysis. While the  indicates the predictability of the criterion variable from the predictor set of variables, the canonical correlations indicate the predictability of each of the extracted component of the criterion set of variables from the corresponding components, extracted from the predictor set of variables. Eigenvalues of the canonical solution indicate the proportion of variance in the predictor set shared with the criterion set of variables on every canonical variate (dimension) extracted. In general, there will be as many canonical variates (dimensions) extracted, as there are variables in the smaller set of either predictor or criterion variables.

          The significance of the canonical correlations can be tested by Wilk's lambda

                                                                       

 

 

Wilk's lambda is computed as a continued product of coefficients of alienation, i.e., for our example

 

 

and

 

The comparison of the obtained and tabulated values for the chi-square indicates that only the first canonical variate is significant.

Eigenvectors

Following the extraction of eigenvalues, the characteristic equation

 

 

will allow us to solve for the characteristic vectors (eigenvectors) associated with each characteristic root (eigenvalue). Thus, for the first eigenvalue () the characteristic equation can be written for the example as

 

 

 

Subtracting the value of the first eigenvalue from the elements located in the principal diagonal of the matrix M

 

 

and postmultiplying with the vector c

 

 

 

allows us to obtain the structural equation for the first component of the predictor set of variables. It should not matter whether you use the top or the bottom equation from the above set of two equations to compute the eigenvectors. Thus, using the top equation

 

 

the homogenous equation relating the  and  coefficients was obtained, as listed below

 

 

The first eigenvector thus can be written in a structural form as

 

 

Using the bottom equation, the  should be identical to the value obtained from the equation on the top

 

and to the homogenous equation relating the  and  coefficients as

 

 

The structural form of the eigenvector is thus confirmed to be

 

 

Since this is a homogeneous set of equations, the C matrices will initially contain a universal set of weights, which as in factor analysis, can be normalized.

Normalization of Eigenvectors

The predictor eigenvectors of canonical analysis are analogous to vectors of beta weights of the regression analysis. In regression analysis we can write Y = bX. In canonical analysis the analogous equation is U = cX. If normalized, the variance of the first predictor eigenvector should be unity. Thus

 

 

or alternatively, realizing that , as

 

 

However, this is obviously not true of our non-standard eigenvector, since

 

 

 

In general, the ipsative products of non-standard canonical eigenvectors will be equal to some number , as

 

 

If normalized, theta should equal 1, as

 

 

Forming a ratio of the above both equations as

 

 

the normalizing constant k can be obtained as

 

For the example,

 

 

Normalizing the structural solution for the first canonical variate as

 

 

gives us a vector of standard canonical weights

 

 

which is identical to the column of canonical weights for the first predictor variate, obtained from a program for canonical analysis, and listed further in our discussion.