Introduction to Matrix Algebra

The summation notation of univariate and bivariate methods of statistical analysis has to be exchanged for the matrix algebra notation of the multivariate methods to facilitate understanding of statistics of more than two variables. The use of matrix algebra notation is a sine qua non requirement for succinct expression of complex multivariate statistical procedures. The main reason behind the use of the matrix algebra to describe multivariate analyses of data is that the matrix algebra notation is concise, avoiding the plethora of summation signs and their associated subscripts and superscripts arising when this subject is described by using the summation algebra notation.

Definitions

A matrix is a rectangular arrangement of numbers. The numbers, contained by the matrix, are referenced by row and column coordinates. Consider a matrix A

written in formal notation as

 

The subscripts of the elements of the above matrix are arranged in the row-column sequences. In general, an element of a matrix is subscripted as where i is the row counter and j is the column counter. The number of rows of a matrix is typically signified by the letter n and the number of columns by the letter k. An element of a matrix thus can be located along the 1...i...n and the 1...j...k continua. The n and k dimensions of a matrix determine its size. The dimensions of the above matrix are 2 by 3.

Vectors and Scalars

Vectors and scalars are special cases of matrices. A vector is a special case of a matrix with either a single row, called a row vector,

or a single column, called a column vector.

Standard notation for vectors is a bold lowercase letter. The convention to denote both vectors and matrices by an uppercase letter is specific to this book. A matrix with a single row and a single column has only a single element.

This element is a scalar number. The ordinary algebra deals with scalars and is sometimes referred to as a scalar algebra.

Nomenclature of Matrices

A special case of a rectangular matrix

 

 

is a square matrix with equal number of rows and columns.

 

The dimension of a square matrix is called an order. The above square matrix has its order equal to 2.

          A symmetric matrix is a special case of square matrix with identical supra and infra diagonal elements

 

 

 

          A skew symmetric matrix is a symmetric matrix with supra and infra diagonal elements of equal magnitude, but opposite sign

 

 

 

          A diagonal matrix has all off diagonal elements equal to zero

 

 

 

          A special case of a diagonal matrix is a scalar matrix where all diagonal elements are equal

 

 

 

          A special case of a scalar matrix is an identity matrix

 

 

 

with all diagonal elements equal to one and all off-diagonal elements equal to zero. Matrices whose elements equal to one are called the unit matrices

 

 

and the matrices filled with zeroes are called the null matrices

 

 

 

A special case of a vector

is a scalar vector

 

with identical elements. Special case of a scalar vector are a unit vector

 

with all elements equal to one and a null vector

 

with all elements equal to zero.

Matrix Definitions

A matrix definition most frequently encountered is called the transpose of a matrix, designated by the letter of the original matrix with attached prime sign. The transpose of a column vector is a row vector. The transpose of a matrix is a matrix with interchanged rows and columns. Consider a matrix

 

Its transpose

             

 

is obtained by writing rows of matrix A as columns in matrix A'.

          Another type of a matrix definition is a triangulation of a matrix. Square symmetrical matrices, as, e.g., matrices of correlations

 

are frequently triangulated to either supra-diagonal

 

or infra-diagonal matrices

 

 

prior to publication of results of correlation analyses to avoid repetitions of identical values. A matrix can be also triangulated into its upper triangular

 

or lower triangular

 

 

parts. A special case of triangulation of a matrix is conversion of the negative elements of a skew symmetric matrix

into zero elements

 

 

Partitioning of a supermatrix into its component submatrices can be also classified as a matrix definition. If a matrix contains several distinct groups of elements, it is called a supermatrix. A supermatrix contains two or more submatrices. Consider a supermatrix

 

containing two submatrices. The above supermatrix can be partitioned into its component submatrices

and

 

A matrix can be augmented by attaching a row or column vector to a matrix or by attaching another matrix adjacent to the matrix row or column margins. To augment a matrix

 

 

by a row vector [5 6] results in augmented matrix

 

 

 

The matrix structure can be decomposed so its elements can be treated as a single variable. The decomposition of the column structure of the matrix A

 

 

results in the row vector

 

The decomposition of the row structure results in a column vector

Characteristics of a Matrix

For every square matrix, there is a unique number called determinant, denoted as |A|. The determinant of a two-by two matrix

 

equals the product of the elements in the principal diagonal minus the product of the off-diagonal elements

For example,

 

The determinant of a matrix determines whether a matrix is invertible. We will discuss matrix inversion in the next chapter. Computation of determinants of square matrices larger than two-by-two is more complicated and should be done by using computer. An alternative notation for the determinant is detA. For the above example we could have written that detA = -2.

          Another unique characteristics of a matrix are sums of its elements, either all, or sums of the elements located in either of the parts of the matrix. The main parts of the matrix are its rows, columns, diagonals, and the areas above and below its principal diagonal. Other unique characteristics of a matrix are sums of squares within these areas. The sum of elements in the principal diagonal of a matrix is called a trace. Thus the trace of a matrix A

equals 1 + 5 + 9. In formal notation this can be written as trA = 15.

Matrix Transformations

The row or column elements of a matrix can be subjected to linear, curvilinear, logarithmic, area, and other types of transformations. Consider a prototypical data matrix

The elements of the above matrix can be transformed into deviation scores

 

by subtracting the arithmetic mean of each column of the matrix X from its constituent elements. The matrix D can be transformed to the matrix of standard scores Z

 

by dividing the elements of the matrix D by the standard deviation of their respective columns. Analogous operations can be also performed on matrix rows.

          The elements of a matrix can be also normalized. Consider a vector [.6 .8]. This vector is considered to be normal or of a unit length, since the sum of its squared elements equals 1.00. In formal language, a vector is normal if the scalar product of the normalized vector with itself equals one. The scalar products of vectors will be discussed in the chapter to follow. To normalize a vector X, divide its each element by a constant c

 

The column normalizing constants of the matrix X equal 7.416 and the normalized matrix X, designated as N, equals

Aside of matrix definitions and transformations, the third broad category of matrix algebra is that which describes operations on matrices, to be discussed in the chapter to follow.