In this explainer, we will learn how to identify the properties of matrix multiplication, including the transpose of the product of two matrices, and how they compare with the properties of number multiplication.
Suppose that we define the matrix and decide to “scale” it by the number 5. To complete this operation, we would simply take every entry of and multiply by 5, giving
Two effects of this operation are easily observed:
- The scaled matrix has the same order as the original matrix (meaning that it has the same number of rows and columns)
- The same operation has been applied to every entry (in this case, multiplication by 5).
Scaling a matrix is a very straightforward operation to understand, as well as being routinely useful when working in linear algebra. Without further exploration, it is believable that this type of operation would be the only type of matrix multiplication that we can define. However, there is another form of matrix multiplication that can be well defined and with properties that are different to either of the two properties of scalar multiplication that are described above. Furthermore, this alternative type of matrix multiplication will be suffused with a range of algebraic properties that can be studied in contrast to the properties of multiplication in conventional algebra. Before we begin studying some of these properties, we will first give the definition of matrix multiplication.
Definition: Matrix Multiplication
Suppose that is a matrix with order and that is a matrix with order such that
Then, the matrix multiplication is a matrix with order , having the following form:
The entries are calculated by the pairwise summation of entries from and as shown:
A common theme in linear algebra, especially for newer students, is that the definitions normally appear to be very abstract and more complicated than concepts that they represent, especially when considering that the majority of matrix calculations are fairly straightforward to perform. To demonstrate that the definition above is not as difficult as it might first seem, we give an example.
Suppose that we have the two matrices
We see that is a matrix with order and is a matrix with order . The matrix must therefore have order and hence be of the form where the entries are values that are currently unknown.
To calculate the entry in the first row and first column of , we highlight the first row of and the first column of :
Then, we multiply together the highlighted entries in order of their appearance, adding them together: . Now we can insert the first entry into the matrix:
Now we calculate the entry of which appears in the first row and the second column, by highlighting the entries in the first row of and the second column of : where we have performed the calculation .
Continuing, we calculate the entry in the second row and first column of , by highlighting the second row of and the first column of : where we have calculated that .
Finally, we highlight the second row of and the second column of : with the final calculation being .
The conditions on matrix multiplication are actually quite restrictive. We know that the matrix product will only exist if has order and has order , meaning that the number of columns in is the same as the number of rows in . If we attempted to reverse the order of the multiplication and calculate , we would then be attempting to combine a matrix with order and a matrix with order . This means that is only well defined if .
For the matrices and above, we can see that is well defined. Although it would not generally be the case, in this instance we also find that is well defined and has order . We have
Even though, rarely, both and are well defined for the two matrices above, we immediately see that for the simple reason that has order and has order . This is a general property of matrix multiplication, which is referred to as the “noncommutative” property.
Theorem: Matrix Multiplication Is Not Commutative
Suppose that is a matrix with order and that is a matrix with order , meaning that and are both well defined. Then generally , which means that matrix multiplication is not “commutative.”
In the case where , this result is obvious because will have order and will have order , meaning that equality is impossible. However, if , then will be a square matrix with order and will also be a square matrix with order . This situation therefore allows the possibility that and are equal, since they have the same order. Although there are some categories of matrices which are commutative under certain conditions, this will generally not be the case.
Example 1: Noncommutativity of Matrix Multiplication
Given that find and .
We will illustrate the calculation of the matrix . Since has order and has order , the resulting matrix will have order .
We calculate the entry of which appears in the first row and the first column: where we have calculated that .
We continue this process for the entry of which appears in the first row and second column: after calculating that .
Next, we move onto the entry in the second row and first column of : which was calculated as .
Finally, we determine the remaining entry: since .
Had we chosen to reverse the order of the matrix multiplication, we would have found that
Clearly , which gives one example of why matrix multiplication is generally not commutative.
Example 2: Noncommutativity of Matrix Multiplication
Consider the matrices and . Is ?
Completing both sets of multiplication gives and
As we can see in this example, .
The two examples above show that generally for two matrices. However, this does not necessarily mean that matrix multiplication is always noncommutative. There are, in fact, categories of matrices which are commutative under matrix multiplication.
Example 3: Possibility of Commutativity in Matrix Multiplication
State whether the following statement is true or false: If and are both matrices, then is never the same as .
Suppose that and are diagonal matrices, which means that every nondiagonal entry must have a value of zero: We can check that and that
In this instance, we see that , meaning that the two matrices above are commutative. The statement in the question is therefore false.
The phenomena above is not unique to the two given matrices and , and we can actually generalize this result to make a statement about all diagonal matrices.
Theorem: Diagonal Matrices Commute
If and are both diagonal matrices with order , then the two matrices commute. In other words, .
We could demonstrate this theorem by picking any two diagonal matrices with the same order, which we did for matrices in the previous example. It is important to note that the theorem above states that and must both be diagonal matrices. For example, consider the two matrices where is a diagonal matrix and is not a diagonal matrix. In this instance, we find that so clearly .
In addition to diagonal matrices, there are other categories of matrices which will always commute with each other, such as pairs of matrices that are simultaneously diagonalizable (this is a topic which should be explored separately!). There are also some very special matrices that commute with every other matrix of a compatible order.
Definition: The Identity Matrix
An “identity matrix,” also known as a “unit matrix,” is a diagonal matrix where all of the diagonal entries have a value of 1. The identity matrix of order is normally denoted as or .
For example, all of the following matrices are identity matrices:
A key property of the identity matrix is that it commutes with every matrix which is of order . For example, consider the identity matrix and the matrix
Then, we could calculate that
Similarly, we could calculate that
We see that , meaning that these two matrices are commutative. Interestingly, this property is actually the result of a much stronger condition that is true of the identity matrices.
Theorem: Effect of the Identity Matrix
Consider the identity matrix and a matrix of order . It is always the case that
A property of the identity matrices is that they leave any matrix unchanged after multiplication, meaning that they must commute with all matrices that have a compatible order. The identity matrix is also used to define another type of matrix that is commutative. We have already seen that two diagonal matrices will be commutative if they are of the same order and that any matrix will be commutative with respect to the identity matrix. However, for a general square matrix , it may be the case that there is another unique matrix with which this matrix commutes.
Definition: Multiplicative Inverse of a Matrix
For a square matrix with order , the matrix “inverse” is the matrix such that
The inverse of a matrix is the matrix that returns the identity matrix when combined with under the operation of matrix multiplication. The idea of a matrix inverse is so central to linear algebra that some of the best mathematicians have created their own methods for calculating these matrices, including Newton, Gauss, Cayley and Hamilton. Although we will not demonstrate any methods for calculating the matrix inverse in this explainer, we should note that the matrix inverse does not exist for all matrices. To be able to understand which matrices have an inverse matrix, we need to study the “determinant” of a square matrix, which is another topic entirely.
Another feature of the matrix inverse is that it is unique if it exists. Therefore, a matrix only has one inverse such that
As we have already seen, the matrix commutes with the matrix inverse under matrix multiplication, whereas two general matrices and would not have this property unless they are both diagonal (or simultaneously diagonalizable, which is beyond the scope of this explainer).
Example 4: Inverse Matrices
Are multiplicative inverses of each other?
If the two matrices are multiplicative inverses of each other, then they should be commutative under matrix multiplication, producing the identity matrix . We can check that and also that
In both calculations, the output is and therefore the two given matrices are multiplicative inverses of each other.
The algebraic properties of the identity matrix and matrix inverses are especially useful when looking to solve problems in linear algebra, often appearing in the solution methods for systems of linear equations. Even though matrices represent arrays of numbers of arbitrary dimensions, their operations can be treated with a deceptive algebraic simplicity.
Example 5: Properties Of Matrix Multiplication with Invertible Matrices
Suppose and is an invertible matrix. Does it follow that ?
Since is an matrix and is also invertible, there must exist a matrix such that where is the identity matrix. We now take the original equation and multiply on the left-hand side by the matrix , giving We know that , giving
We also know that there is no change to the matrices and when combining with the identity matrix under matrix multiplication, meaning that and that . In other words, we have indeed shown that
Although we have explored some of the fundamental properties of matrix multiplication, there is another important result that we will cover in this explainer, which can simplify many calculations in linear algebra. We will cover this first by way of example and then we will state a general theorem.
Example 6: Distributive Property of Matrix Multiplication
Given that is it true that ?
We first calculate
Next, we calculate
As we might have expected, we do find that .
The phenomenon that we have just witnessed is known as the “distributive” property of matrix multiplication with respect to matrix addition. This property is summarized by the following theorem.
Theorem: Matrix Multiplication is Distributive
Suppose that is a matrix with order and and are matrices with order . Then, matrix multiplication is distributive with respect to matrix addition; that is,
In the statement of this theorem, it was necessary to specify the orders of the three matrices, to ensure that all operations were possible. If and had been of different orders, then it would have been impossible to define the matrix addition . Also, if did not have the same number of columns as the number of rows in and , then we could not have calculated or . When working with matrices, it is very common for the orders to be specified in any definition or theorem because not doing so might mean that the involved operations cannot be completed.
Example 7: Distributive Property of Matrix Multiplication
State whether the following statement is true or false: If is a matrix and and are matrices, then .
The matrices and have the same order of , which means that can be calculated, with the resulting matrix also having the order . The multiplication is therefore between a matrix with order and a matrix with order , meaning that this is well defined. The resultant matrix has order .
For the same reasons, the multiplication produces a matrix with order , which is also true for the multiplication . Every matrix operation is therefore well defined, with the result being a matrix of order . Since matrix multiplication is always commutative with respect to addition, it is therefore true in this case that
There are many more properties of matrix multiplication that we have not explored in this explainer, especially in regard to transposition and scalar multiplication. For square matrices in particular, matrix multiplication is a key area of consideration when studying crucial concepts such as the determinant. One of the most curious aspects of matrix multiplication is that the definition is apparently quite complicated, at least for people who are new to linear algebra. In contrast to the definition, the algebraic properties of matrix multiplication are fairly straightforward, being largely similar to multiplication in conventional algebra except for the key difference around the area of commutativity. There are also many types of matrix which have special algebraic properties in regard to multiplication, and we have already covered diagonal matrices, the identity matrix, and inverse matrices. In particular, symmetric matrices and simultaneously diagonalizable matrices all have enhanced algebraic properties which make them interesting to study.
- Matrix multiplication is generally not commutative; that is, .
- If and are both diagonal matrices with the same order, then .
- The identity matrix commutes with all matrices of the same order, leaving the original matrix unchanged; that is, .
- For a square matrix of order , there may exist a unique inverse matrix such that .
- Matrix multiplication is distributive with respect to matrix addition; that is, .