Matrix Algebra
Operations
In this chapter, we will discuss the
elementary operations of addition, subtraction,
multiplication, and inversion, a matrix algebra analogue of
division of scalar numbers. Subsequently, we will turn our
attention to powers of matrices and to operations involving
scalar numbers and matrices, scalar numbers and vectors, and
vectors and matrices.
The operations on matrices differ from
similar operations of scalar algebra in several respects.
The matrix algebra operations, in general, are not commutative
and attention must be paid to whether the matrices are conformable
with respect to the intended operation. Also, it must be
noted whether the matrix operation pertains to matrix
elements or to matrices.
Addition and Subtraction of
Matrix Elements
Addition
of Matrix Elements
To add elements of matrices A and B and
store the result as a matrix C,
elements of matrix A
are added to
their corresponding elements in matrix B and stored as
elements of matrix C. Obviously, all three matrices must be
of the same order or size. Notice that the plus sign in the above
equation is enclosed in parentheses to indicate addition of
matrix elements, as contrasted with addition of matrices, to
be discussed in section to follow. An example of addition of
matrix elements is schematized below
and illustrated as
Subtraction
of Matrix Elements
The operation of subtracting matrix
elements can be schematized as
and illustrated as
In sum, the
above matrix
elements can be added and subtracted if and only if the
matrices are
of the same order (identical in the number of rows and
columns). Matrices upon which an operation is permissible
are said to conform to the operation.
Addition and Subtraction of
Matrices
Textbooks
on matrix algebra, routinely describing major and minor vector products, do not
suggest analogical operations for the major and minor sums and differences of
summands, minuends, and subtrahends. These operations are easy to imagine and
are not discussed because most of their potential applications can be as well
accomplished by unit vectors multiplications. However, on close scrutiny, matrix
algebra operations of addition and subtraction (of vectors, not elements of
vectors, of matrices, not elements of matrices) can be used for concise
expression of several key algorithms of statistical theory and theory of
probability.
The novel matrix algebra
operations were first formulated in the following two
articles:
Krus, D.J., & Ceuvorst, R. W.
(1979)
Dominance, information, and hierarchical scaling of
variance space. Applied Psychological Measurement, 3,
515-527.
Krus, D.J., & Wilkinson, S.M.
(1986)
Matrix differencing as a concise expression of
variance. Educational and Psychological Measurement,
46, 179-183.
Addition
of Matrices
To add two matrices A and B and store
the results in matrix C,
the number of
columns in matrix A must
equal the number of rows in matrix B, in another words, the matrices must be
conformable
to matrix addition. The resulting matrix C will have the
number of rows of the first matrix and the number of
columns
of the second matrix. For example, if matrix A is a 3x2
matrix and matrix B is a 2x3
matrix, the resulting matrix will be a 3x3
matrix.
The schematic representation of matrix
addition is
illustrated as
Subtraction
of Matrices
To subtract two matrices A and B and
store the results in matrix C
again, the matrices must be conformable. The number of
columns in matrix A must
equal the number of rows in matrix B. The resulting matrix C will have the
number of rows of the first matrix and the number of
columns
of the second matrix. The schematic representation of
matrix subtraction is shown below
illustrated as
Note that the
first matrix is a 2x3
matrix and the second matrix is a 3x2
matrix, the resulting matrix is a 2x2
matrix.
Addition and Subtraction of
Vectors
The operation of addition and subtraction can be also
performed on vectors.
Addition
of Vectors
Major
Addition
Addition of A
Column Vector and A Row
Vector
Consider a column
vector
X
It's major
addition
can be symbolically represented as
and illustrated as
Note that the
first matrix is a 3x1
matrix and the second matrix is a 1x3 matrix, the resulting matrix is a
3x3
matrix.
Minor
Addition
Addition
of A Row Vector and A Column Vector
The minor
addition of the column vector X
can be symbolically represented as
and illustrated as
Note that the first matrix is a
1x3 matrix and
the second matrix is a 3x1 matrix, the resulting matrix is a scalar.
Subtraction
of Vectors
Major
Subtraction
Subtraction of A
Column Vector and A Row Vector
Consider a column
vector
X
It's major
subtraction
can be symbolically represented as
and illustrated as
Minor
Subtraction
Subtraction
of A Row Vector and A Column Vector
The minor
subtraction of the column vector X
can be symbolically represented as
and illustrated as
Addition
of A Vector and A Matrix
Since the matrix and the vector must be
conformable to addition, a matrix can be added only to a row
vector and only a column vector can be added to a matrix.
The same is true of subtraction of matrices and vectors.
Column Sums
The addition of a row zero
vector and a matrix
can be symbolically represented as
and illustrated as
results in a column sum of matrix
elements (9, 12).
Notice that the row vector and the matrix are conformable. The
vector is a 1x3 vector and
the matrix is a 3x2 matrix, the resulting matrix is a 1x2
row vector.
Row Sums
The addition of a matrix and a
column zero vector
can be symbolically represented as
and illustrated as
results in row sums of matrix
elements (3, 7, 11).
Notice that the matrix and the column vector are conformable.
The matrix is a 3x2
matrix and
the vector is a 2x1 matrix, the resulting matrix is a 3x1
column vector.
Multiplication
of Matrix Elements
To multiply elements of two matrices A
and B and store the results in matrix C
each element of matrix A is multiplied
with its corresponding element in matrix B and the result is
stored in matrix C. To perform this operation, both matrices
have to be either of the same size, or one matrix has to be
a scalar matrix. The schematic representation of
multiplication of matrix elements is
illustrated by a special case of
multiplication of a matrix by a scalar number as
in the special case of multiplication
of a matrix by a constant it is customary to omit the (.)
sign.
Multiplication
of Matrices
The matrix multiplication is analogous
to matrix addition, and, as the matrix addition, this
operation is not commutative, unless both matrices are
symmetric. Also, to premultiply
matrix B by matrix A
the matrices must be conformable
to matrix multiplication. The number of
columns in matrix A must
equal the number of rows in matrix B. The product matrix C will have the
number of rows of the first matrix and the number of columns
of the second matrix. The schematic representation of matrix
multiplication is
illustrated as
The postmultiplicatin
of matrix B by matrix A
using the same numerical example, is
illustrated as
Premultiplication of a matrix A by a diagonal matrix
T
is equivalent to scalar multiplication
of each row element in the matrix by the same row element of
the diagonal matrix. In schematic representation
The following example illustrates rearrangement of
the columns of a matrix
Division
of Matrix Elements
To divide elements of two matrices A
and B and store the results in matrix C
corresponding elements of A and B
matrices form a fraction, stored in C. All three matrices
must be of the same size, or the divisor must be a scalar
number. The division sign is enclosed in parentheses to
indicate division of matrix elements. An example of division
of matrix elements is schematized below
and illustrated, using as an example a
division of a matrix by a scalar number
Inverse
The analog of matrix division is
multiplication by a reciprocal. The reciprocal of a matrix
is called its inverse. Only square matrices can be inverted,
however, some square matrices, called singular
matrices, do not have inverses. The inverse of a matrix
is denoted by a -1 superscript.
To invert a matrix, first,
we must determine
that the matrix is invertible by computing its determinant. If the determinant is not a zero,
the matrix is invertible. This is analogous to the
restriction on the division of scalar numbers that cannot be
divided by a zero. Consider a special case of a two by two
matrix A
with determinant equal to
ad-bc. For
the special case of the two-by-two matrices, the inverse
of the above matrix can be calculated by
switching elements in the principal diagonal,
changing signs of the off-diagonal
elements,
and
by dividing all elements by the determinant.
Consider
a matrix A
First, we must determine whether this
matrix is invertible. The determinant is computed as
(1)(4)-(2)(3) and equals -2. This determinant does not equal
zero. It means that the matrix is invertible.
To invert a matrix, switch elements in
the principal diagonal,
change signs of the off-diagonal
elements,
and divide all elements by the
determinant
Thus, the matrix inverse equals
Analogous to scalar multiplication of a
number by its reciprocal
an identity matrix I equals
this analogy also suggests the nature
of matrix singularity.
In arithmetic, the only number
without a reciprocal is 0. In matrix algebra, the singular
matrices, i.e., matrices with a zero determinant, do not
have an inverse. A numerical illustration of the above
equation is
The discussed procedures for matrix
inversion apply only to two-by-two matrices. The inverse of
a matrix larger than a two-by-two matrix is difficult, best
done with the help of a computer.
Powers
of Matrix Elements
The squares of the matrix elements are
written as
In schematic representation
A numerical example is
A matrix to a fractional power
provides
for obtaining roots of matrix elements and a matrix to a
fractional negative power
for obtaining reciprocals of roots of
matrix elements.
Powers
of Matrices
The square of a matrix is a
multiplication of a matrix by itself
In schematic representation
this operation is illustrated as
the results of the illustrative example
can be compared with the previous results to stress that a
power of a matrix
is not equal to power of matrix elements
.
