CHAPTER 4
ELEMENTS OF MATRIX ALGEBRA

Matrix algebra began as the study of systems of linear equations. Today, it provides a basic framework for the study of modern algebra, calculus of several variables, and differential equations.  Although algebraic operations on matrices may seem a bit abstract, they do serve a great purpose by simplifying the mathematical notations and operations of the underlying models of economic problems.  Chapters 6 through 8 of this book employ matrix algebra in optimizing problems, where the function has two or more independent variables. Chapters 9 and 10 utilize the concept of rank and linear dependence as well as matrix algebra in solving linear programming problems and linear economic models. Chapter 11 makes an extensive use of concepts developed in this chapter and chapter 5, such as eigenvalues and eigenvectors, in solving systems of first order differential equations.

TOOL BOX
Mathematica Commands			Function
a = {5, 3, 2}					a is a 3x1 column vector or a 1x3 row vector.
B = {{3, 4, 5}, {4, 5,1}, {-1, 2,4}}		B is a 3x3 matrix with the given elements
MatrixForm	[B]				Writes B in matrix form
MatrixForm [A.B]  				Pre-multiplies B by A and writes the product in matrix form	
Det[B]						Finds determinant of matrix B
Transpose[B]					Transposes matrix B
Inverse[B]					Inverses matrix B 
Tr[B]						Finds trace of matrix B	
IdentityMatrix[n]				Generates a nxn identity matrix
Dot [a ,  b]					Finds dot product a & b vectors of dimension 3 or higher
Cross [a,  b]    					Finds cross product a & b vectors                                    

Note:  To generate a matrix in Mathematica, take the following steps:

Click on "File", click on "Palettes", and click on "2 Basic Calculations" to open the basic calculation window.  In the new window,  click on "List and Matrices", click on "Creating Lists and Matrices", and click on the matrix or vector icon.  To expand a matrix or a vector, click on one element of matrix or vector on the screen and push "Ctrl" and "Enter" keys to add a row, and push "Ctrl" and "," keys to add a column.

I. Algebra of Matrices: The MatrixForm, Det, Transpose, Inverse, and Tr Commands                     	
Matrix:  A matrix is a rectangular array of elements arranged in rows or columns as 

A = (({{a11, a12, ., ., ., ., ., ., ., a1n}, {a21, a22, ., ., ., ., ., ., ., a2n}, {., ., ., ., ., ., ., ., ., .}, {., ., ., ., ., ., ., ., ., .}, {am1, am2, ., ., ., ., ., ., ., amn}}))
A has mxn elements arranged in m rows and n columns, and is said to be of order m by n. The elements in the (i^th) row and (j^th) column is represented by aij.  Matrices provide a powerful way of organizing and manipulating data.  In many instances a large amounts of information can be summarized in matrix form.  Matrices are usually denoted by bold capital characters.

Example:  A car manufacturer is producing three models of cars, sedan (S), full-size (F), and Vans (V).  Total production (TP) of the three models (in thousands) for the years 1994-1998 is summarized in a 5x3 matrix, 
TP = (Overscript[(({{45, 32, 28}, {52, 38, 31}, {66, 41, 39}, {72, 42, 45}, {78, 45, 52}})), (({{S      , F  , V}}))]), where the columns represent the models and the rows represent the years, with the first row representing 1994.

Vectors:  A matrix of order 1xn is called a row vector.  For example,  b = (({{b1, b2, b3}})) is a 1x3 row vector.  A matrix of order mx1 is a column vector.  For example, c = (({{c1}, {c2}, {c3}}))is a 3x1 column vector.  With these definitions, a matrix can also be viewed as a set of column vectors or row vectors.  That is the matrix A above can be written as A = (({{a1, a2, ., ., ., an}})), where a1 = (({{a11}, {a21}, {.}, {.}, {am1}})), a2 = (({{a12}, {a22}, {.}, {.}, {am2}})), and so on.

In physical science, the term vector is used to indicate a quantity (such as force and velocity) that has both magnitude and direction.  In such cases, a vector is often represented by an arrow or directed line segment.  The length of the arrow represents the magnitude and the tip of the arrow points in the direction of the vector.  In business and economics, a vector may represent a collection of data for one variable over time (a table of time series data) or it may represent a cross section collection of data on a variable at one time.  The k elements of a row or column vector a can be viewed as the coordinates of a point in a k-dimensional space.  Vectors are usually denoted  by bold lower case character. 

Example:  Price of the sedan cars (in thousand dollars) for 1994-1998 may be represented by a 1x5 vector pt = (({{9.2, 9.6, 1.2, 1.8, 11.2}})).  This is an example of Time-series data on one variable presented in a vector form.  The prices of the three models of cars for the 1994 may be represented by a 1x3 vector pc = (({{9.2, 11.4, 18.75}})).  This is an example of a cross-section data presented in a vector form. 
		
Algebra of Matrices:  The algebra of matrices utilizes the  following rules:

Equality of Matrices:   Two matrices A and B are said to be equal when they are of the same order and all the corresponding elements of A and B are equal.  

Example:  Find x and y in a way that the two matrices A = (({{2, -1}, {3, 4}})) and B = (({{2, -1}, {.5x, y - x}})) are equal.
Solution:  For A and B to be equal .5x = 3 and y - x = 4 should be satisfied.  Solving these two equations together gives x = 6 and y = 10.  

Addition of  Matrices:  If A and B are of the same order, then A + B is obtained by adding the corresponding elements of A and B.

Example:  Two matrices A and B are given below.  Find A + B. 
A = (({{3, -1}, {0, 10}})),   B = (({{2, 4}, {-7, 5}}))
Solution:  Adding  up the corresponding elements of the matrices gives (({{3 + 2, -1 + 4}, {0 - 7, 10 + 5}})) = (({{5, 3}, {-7, 15}})).  To solve the problem using Mathematica, define the matrices, add them, and write in matrix form using "MatrixForm" command
Clear[A,  B]
A = (({{3, -1}, {0, 10}}))  ;   B = (({{2, 4}, {-7, 5}}));

MatrixForm[A + B]
(({{5, 3}, {-7, 15}}))

The Difference of  Matrices:If A and B are of the same order, then A - B  is obtained by subtracting the corresponding elements of B from A.  

Example: For the two matrices A and B of the preceding example find A - B.
Solution:  Subtracting the corresponding elements of the two matrices gives (({{3 - 2, -1 - 4}, {0 + 7, 10 - 5}})) = (({{1, -5}, {7, 5}})).  
Using Mathematica, the problem can be done as, 
Clear[A,  B]
A = (({{3, -1}, {0, 10}}))  ;   B = (({{2, 4}, {-7, 5}}));

MatrixForm[A - B]
(({{1, -5}, {7, 5}}))

Scalar Multiplication:If k is a scalar, the scalar multiplication of A is obtained by multiplying each ele-ment of A by k.  

Example: For the matrix A =  (({{3, -1}, {0, 10}})) find kA.
Solution:  Multiply each element of the matrix A by k to get (({{3  k, -k}, {0, 10  k}})).  
In Mathematica,
Clear[A]
A = (({{3, -1}, {0, 10}}));

MatrixForm[k A]
(({{3 k, -k}, {0, 10 k}}))

Example:  Unit Cost 
A manufacturer has two plant p1 and p2.  The cost of inputs per unit of labor, capital, and material in each plant are shown in the following matrix, C = (Overscript[(({{23, 35}, {42, 38}, {33, 40}})), (({{p1, p2}}))]), where the columns are representing the plants 1 and 2 and the rows 1-3 are representing labor, capital, and material unit costs, respectively.  It is expected that, the next year's unit costs will rise by 6%.  Find the expected unit cost matrix for the next year.
 Solution: Multiply all elements of the cost matrix by 1.06 to get (({{24.150000000000002`, 36.75`}, {44.1`, 39.9`}, {34.65`, 42.`}})) .  In Mathematica,
cost = (({{23, 35}, {42, 38}, {33, 40}}));
MatrixForm [1.05 cost]
(({{24.150000000000002`, 36.75`}, {44.1`, 39.9`}, {34.65`, 42.`}}))

Matrix Multiplication:  If A is of order mxn and B is of order nxp, then the dot product of A.B is a matrix of order mxp, where the ijth element is obtained by multiplying the elements of the ith row of the first matrix by the corresponding elements of the jth column of the second matrix and summing over all terms. For this to be possible, it is clear that the number of columns in the first matrix (n) should be equal to the number of rows in the second matrix (n). If that is the case, the matrices are said to be conformable for multiplication. 

Example:  Find the dot product of A =  (({{a11, a12}, {a21, a22}})) and B =  (({{b11, b12}, {b21, b22}})).
Solution:  The product is a 2x2 matrix with elements as (({{a11 . b11 + a12 . b21, a11 . b12 + a12 . b22}, {a21 . b11 + a22 . b21, a21 . b12 + a22 . b22}})).  
To do the same multiplication using Mathematica, note that, in Mathematica the dot product of matrices is done using "." rather than "*" or space.  The latter symbols are reserved for a  special multiplication which will be explained in the next section.
Clear[A, B]
A = (({{a11, a12}, {a21, a22}}));  B = (({{b11, b12}, {b21, b22}}));
MatrixForm[ A . B ]
(({{a11 b11 + a12 b21, a11 b12 + a12 b22}, {a21 b11 + a22 b21, a21 b12 + a22 b22}}))
  
Example: Find A.B for A and B given as A = (({{3, -1}, {0, 10}})) B = (({{2, 4}, {-7, 5}})).
Solution:  Let's call the product C, C = A.B.  To find the c11 element of the product matrix (element in the first row first column), multiply the elements of  the first row of A with the elements of the corresponding elements of the first column of B and add the products.  That is, c11 = 3x2 + (-1)(-7) = 13.  To find the c12 element of the product matrix (element in the first row second column), multiply elements of  the first row of A with the corresponding elements of the second column of B and add the products.  That is, c12 = 3x4 + (-1)(5) = 7.  Continue to find the other elements.  The resulting matrix is  (({{13, 7}, {-70, 50}})).  
Using Mathematica, the product is found as
Clear[A,  B]
A = (({{3, -1}, {0, 10}}));     B = (({{2, 4}, {-7, 5}}));

MatrixForm[A . B]
(({{13, 7}, {-70, 50}}))
It is important to note that not all the laws of ordinary scalar algebra hold for matrix algebra. The following are some of the laws of matrix operations.

The Commutative law of addition holds. If A and B are of the same order, A + B = B + A.
The Associative law of addition holds. That is, (A + B) + C =A + (B + C).
The Commutative law of multiplication does not hold for product of matrices, except in the case of some special square matrices. That is, in general A.B ≠ B.A.
The Associative law of multiplication holds. That is, (A.B).C = A.(B.C).
The Distributive law holds. That is, A.(B + C) = A.B + A.C, and(B + C).A = B.A + C.A.
The Distributive law of scalar multiplication holds. That is,k(A + B) = kA + kB, and (v + k)A = vA + kA.

Example: Input Requirement
A manufacturer is producing three products x, y, and z.  Each unit of x requires 5 units of labor and 1 unit of capital. Each unit of y requires 2 units of labor and 4 units of capital.  Each unit of z requires 3 units of labor and 2 units of capital.  Write the matrix of input requirement.  If the manufacturer decides to produce 50 units of x, 20 units of y, and 80 units of z how many units of labor and capital are needed.
Solution:  The matrix of input requirements per unit of output (IR) is, IR = (({{5, 2, 3}, {1, 4, 2}})).  The first row of the matrix shows labor requirements per unit of outputs and the second column shows the capital requirements per unit of outputs.  The output vector is Q = (({{50}, {20}, {89}})).   Therefore, the total input requirement can be found by finding the dot product of the two matrices (({{5, 2, 3}, {1, 4, 2}})).(({{50}, {20}, {89}})) = (({{530}, {290}})).  That is, 530 units of labor and 290 units of capital.  Using Mathematica, define the matrices and find the product.
Clear[IR,  Q]
IR = (({{5, 2, 3}, {1, 4, 2}}));		Q = (({{50}, {20}, {80}}));

MatrixForm[IR . Q]
(({{530}, {290}}))
Note that, IR is a 2x3 matrix and Q is a 3x1 vector.  The matrices are conformable for multiplication and the product is a 2x1 vector.   

Example:  Total Revenue
A firm operating in three locations manufactures and sells four products (A, B, C, and D), which are sold at market prices (Subscript[P, A]) = 8, (Subscript[P, B]) = 22, (Subscript[P, C]) 13, and (Subscript[P, D]) = 16.  Location 1 supplies 350 units of A and 598 units of D.  Location 2 supplies 960 units of A, 1020 units of C, and 560 units of D.  Location 3 supplies 2040 units of B and 975 units of D.  Write the total revenue of the producer in matrix form and find total revenue for each location and the total revenue of the firm.
Solution:
Clear[TR, P]
TR=(({{350, 0, 0, 598}, {960, 0, 1020, 560}, {0, 2040, 0, 975}}))
P=(({{8}, {22}, {13}, {16}})) 
MatrixForm[TR . P]
{{350, 0, 0, 598}, {960, 0, 1020, 560}, {0, 2040, 0, 975}}
{{8}, {22}, {13}, {16}}
(({{12368}, {29900}, {60480}}))
Total Revenue of the Firm = 12368 + 29900 + 60480 = 102748

Direct Multiplication of Matrices:   There are many cases in which a column or a row of a matrix needs to be scaled up or down.  In these cases direct or "*" multiplication is a useful tool.  Direct product of two matrices is a matrix with the elements that are the product of the corresponding elements of the two matrices.  

Example:  Find the direct product of A = (({{a1, b1}, {c1, d1}})) and B = (({{a2, b2}, {c2, d2}})).
Solution:  The direct product of the two matrices is (({{a1a2, b1b2}, {c1c2, d1d2}})).   
Using Mathematica, direct product of the two matrices is,
MatrixForm[(({{a1, b1}, {c1, d1}}))*(({{a2, b2}, {c2, d2}}))]
(({{a1 a2, b1 b2}, {c1 c2, d1 d2}}))

Example:  Find the direct product of the A = (({{3, 1}, {7, 2}})) and B = (({{5, 2}, {1, -3}})).
Solution:  The direct product of the two matrices is a matrix with the elements equal to the product of the corresponding elements of the two matrices, (({{15, 2}, {7, -6}})).  
Using Mathematica,
MatrixForm[(({{3, 1}, {7, 2}}))*(({{5, 2}, {1, -3}}))]
(({{15, 2}, {7, -6}}))

Example:  Input-Output Models
The input-output table of a two-sector economy is given as,
                                To Industry:
({{From Industry, 1, 2, Consuption, Total}, {1,  15 ,  10 , 25, 50}, {2, 20, 8, 12, 40}})
find the input-output coefficients.
Solution:  The input-output model is a simplified descriptive model of the economy, describing the interrelationships within the producing sectors (here, industries 1 and 2) and the relationship between producing and consuming sectors of the economy. (Input-Output models are studied in Chapter 10).  In these models, every industry in  the economy buys variable inputs from other industries and sells variable inputs to other industries and final goods to the consuming sectors.   The main objective of the input-output analysis is to determine the level of output that each industry should produce in order to satisfy the total demand for the product of the industry in the economy.  
The inter-industry matrix (yellow in the table above) represents the quantity of goods sold by industry i to industry j.  The last column in the table represents the total goods produced by the sectors.  For practical purposes, it is customary to convert the inter-industry matrix of the table to input-output coefficients by dividing each element to corresponding total output.  This can be done by finding the direct product of the inter-industry matrix and a conversion matrix.  The conversion matrix here is a matrix which its rows are the inverse of the total outputs
(({{15, 10}, {20, 8}}))*(({{(1/50), (1/40)}, {(1/50), (1/40)}})) = (({{.30, .25}, {.40, .20}})). 
Using Mathematica, 
N[MatrixForm[(({{15, 10}, {20, 8}}))*(({{1/50, 1/40}, {1/50, 1/40}}))]]
(({{0.3`, 0.25`}, {0.4`, 0.2`}}))
Therefore, the input-output table with the input-output coefficients can be written as, 
                                To Industry:
({{From Industry, 1, 2, Consuption, Total}, {1,  .30 ,  .25 , 25, 50}, {2, .40, .20, 12, 40}})

Transposition: One operation on matrices that has no equivalent in scalar algebra is called transposition. The transpose of A, denoted A' or (A^t), is obtained from A by interchanging rows and columns.  That is, the first row of A becomes the first column of A', the second row of A becomes the second column of A' and so on.  

Example:  Find the transpose of the following matrix, A =  (({{a11, a12}, {a21, a22}, {a31, a32}})).
Solution:  Change the columns and rows of A to find A' as, A' = (({{a11, a21, a31}, {a12, a22, a32}})).  
In Mathematica, use the command Transpose[A] as follows,
Clear[A]
A =(({{a11, a12}, {a21, a22}, {a31, a32}})); \(\(
\)\(MatrixForm[Transpose[A]]\)\)

Null (({{a11, a21, a31}, {a12, a22, a32}}))

Example: Find the transpose of the following matrix, A = (({{3, 2}, {0, 1}})).
Solution:  The transpose is A' = (({{3, 0}, {2, 1}})).
Using Mathematica, 
Clear[A]
A = (({{3, 2}, {0, 1}}));
MatrixForm[Transpose[A]]
(({{3, 0}, {2, 1}}))
Transposition is often used in matrix multiplication to make two matrices conformable for multiplication.  For example if A is a matrix of order 3x2 and B is a matrix of 4x2, B can't be pre-multiplied by A.  However, transposing B turns it to a 2x4 matrix and make it conformable for pre-multiplication by A.  The product of A.B' will be a 3x4 matrix. 

Example: Total Spending                                                                                                                                                              
A consumer selects a basket of goods containing three items with the prices (P) and quantities (Q) given as P =  (({{2, 4, 11}})), Q = (({{5, 27, 88}})), respectively.  Find the total spending (TS) on the goods.
Solution:  P and Q are given as two 1x3 row vector. To find P.Q,  Q should be transposed to a column vector of 3x1.  That is, the total spending TS = P.Q' = (({{2, 4, 11}}))(({{5}, {27}, {88}})) = 1086.  Note that, the product of two vectors that generates a scalar number is called the inner product of the two vectors.  The inner product and two other types of vector multiplications are discussed in the section titled "Products of Vectors" in this chapter.
Using Mathematica, the inner product of the two vectors is,
Clear[Q,  P,  TS]
Q = (({{2, 4, 11}}));       P = (({{5, 27, 88}}));
TS  =  P . Transpose[Q]

{{1086}}

The theorems of the transposition of matrices are given below, without the proofs. Students are required to verify them using numerical values for A, B, and C (Note: A'  stands for the transpose of A).

(A')' = A
(A + B)' = A' + B'
(A.B)' = B'.A'
(A.B.C)' = C'.B'.A'
(A.B.C)' = [(A.B).C]' = C'.(A.B)' = C'.B'.A'

Trace: The Trace of a square matrix is defined as the sum of the elements in the principal diagonal, ∑(Subscript[a, ii]). The trace of A is denoted by tr(A). 

Example:  Find the trace of A = (({{a11, a12, a13}, {a21, a22, a23}, {a31, a32, a33}})).
Solution:  Add the diagonal elements,  tr(A) = a11 + a22 + a33.
In Mathematica, use the command Tr[A] to find the trace.
Clear[A]
A = (({{a11, a12, a13}, {a21, a22, a23}, {a31, a32, a33}}));
\(\(Tr[A]\)\(
\)\)
a11 + a22 + a33
Alternatively, you can use the following command
Sum[A[[i,  i]], {i,  3}] 
a11 + a22 + a33

 Example: Find the trace of  A = (({{5, -1, 0, 3}, {2, 9, 21, 3}, {-5, 7, 9, 11}, {5, 4, 6, 12}})).
Solution: The sum of the elements in the principal diagonal is 45. 
Using Mathematica,
Clear[A]
A = (({{5, -1, 0, 3}, {2, 9, 21, 3}, {-5, 7, 9, 11}, {5, 4, 6, 12}}));
Tr[A]
35
Alternatively, you can use the command
Sum[A[[i, i]], {i, 4}]
35

Inverse Matrix:   Given matrix A, if a matrix A-1 exists such that AA-1 = A-1A = I,  then A-1 is called the inverse  of A.   Here, I is a unit matrix or identity matrix.  A unit matrix or identity matrix is a square matrix of order nxn with "1" in the principal diagonal and zeros everywhere else.   In order to establish the matrix A-1, we need a few definitions.

Determinant:  Associated with any square matrix A, there is a scalar quantity called the determinant of A,  denoted  by Det[A] or |A|.  This quantity is obtained by summing various products of the elements of A.  

Example:  Find the determinant of A= (({{a11, a12}, {a21, a22}})).
Solution:  Determinant of a 2x2 matrix is the difference between the product of the diagonal and off-diagonal elements.  That is, Det[A] = a11a22 - a21a12.
In Mathematica, use the command Det[A] to find the determinant.
Clear[A]
A= (({{a11, a12}, {a21, a22}}));
Det[A]
-a12 a21 + a11 a22

For higher order matrices, finding determinants require more calculation.   There are several ways to find determinant of higher-order matrices.  One way, called column expansion is as follows:
1- Take a11 element of the matrix A, delete the corresponding row and column, and form a new matrix (A11) with the remaining elements of the matrix.   A determinant  formed  by  deleting a row and a column corresponding to one element of  a matrix  is  called principal minor.  
2- Take a21 element of the matrix A, delete the corresponding row and column, and form a new matrix (A21) with the remaining elements of the matrix.
3- Continue this for all the elements of the matrix A in column one.
4- Det[A] = a11Det[A11]- a21Det[A21] + a31Det[A31] .........

This was a column expansion.  Note that, the sign of the (Subscript[a, ij]) coefficients alternates, beginning  with positive.  The sign of the (Subscript[a, ij]) coefficients follows the sign  convention (-1)i+j, where i+j is the sum of subscripts in (Subscript[a, ij]). That is, for (Subscript[a, 11]) the sign is (-1)2= +1 and for (Subscript[a, 32]) the sign is (-1)5 = -1.  

If you start with (Subscript[a, 11]) and continue with selecting (Subscript[a, ij])s in the first row rather than the first column and follow the steps above, the method is called row expansion.  Both row and column expansion will result in the same number for determinant.  The question of which one to choose is answered by the magnitude of the numbers in the first row or first column.  Obviously, numbers such as zero and one easier to use in numerical multiplication than numbers of larger magnitude.


Example:  Find the determinant of B = (({{3, 1, 0}, {2, 5, 6}, {6, 3, 1}}))
Solution:  Using the column expansion, the determinant can be found as:  Det[B] = 3(({{5, 6}, {3, 1}})) - 2(({{1, 0}, {3, 1}})) + 6(({{1, 0}, {5, 6}})) = 3(5x1 - 3x6) -2(1x1 - 3x0) + 6(1x6 - 5x0) = -5.  However, the elements such as zero and one in the first row make it easier to find the determinant using the row expansion method.  The row expansion would be:  Det[B] = 3(({{5, 6}, {3, 1}})) - 1(({{2, 6}, {6, 1}})) + 0(({{2, 5}, {6, 3}})) = 3(5x1 - 3x6) -1(2x1 - 6x6)  = -5.  In Mathematica, use the command "Det" to find the determinant,       
Clear[B]
B = (({{3, 1, 0}, {2, 5, 6}, {6, 3, 1}}));
Det[B]   

-5
In physical science the absolute value of the determinant of a 2x2 matrix A represents the area of the parallelogram formed by the columns of A.  That is, for A = (({{a, b}})) = (({{4, 3}, {1, 2}})) the area of the parallelogram formed by vectors a = (({{4}, {1}})) and b = (({{3}, {2}})) is (4)(2) - (1)(3) = 5.  The column vectors of a 3x3 matrix form a solid that is a parallelotope.  The determinant of the matrix will be the volume of this solid.

Properties of Determinants: The following properties of determinants are given without proofs.  Students are required to verify them by using numerical examples.
1-   Det[A'] = Det [A]
2-   Det[A.B] = Det[A].[B]
3-   Interchanging any two columns (or rows) of A changes the sign of the determinant of A.
4-   If every element of a row (or a column) of A is multiplied by a scalar k to give a new matrix B, then Det[B] = kDet[A].
5-   If every element of an nth order matrix A  is multiplied by k to give a new matrix B, then Det[B] =  (k^n)Det[A]. 
6-   A determinant is unaltered in value when to any row (column) a constant multiple of any other row (column) is added.  This property can be used to cut the number of arithmetic operations for finding the determinant.

Co-factors:   cij = (-1)i+j Det[(Subscript[A, ij])]s are called co-factors (coefficients).  Hence, c11 = Det[A11], c12 = -Det[A12], c13 = +Det[A13],  and so on.           

Inverse of a Matrix:   To find the inverse of a matrix A, take the following steps.
1.   Find co-factors of A and  form a new matrix so that its elements are corresponding co-factors of A.
2.   Transpose the new  matrix  and call it  adj A, the adjoint of A. 
3.   Divide all the elements of adj A  by determinant of A  to get A-1.

Note that, step 3 is possible only if Det[A] ≠ 0, in which case, the  matrix A is said to be nonsingular. When Det[A] = 0, the matrix is called a singular matrix, and the inverse does not exist.

Example:  Find the inverse of the A = (({{3, 1, 0}, {2, 5, 4}, {6, 3, -1}})).
Step1: Find the co-factors,
  c11 = Det(({{5, 4}, {3, -1}})) = -17		 c12 = - Det(({{2, 4}, {6, -1}})) = 26		 c13 =  Det(({{2, 5}, {6, 3}})) = -24

  c21 = - Det(({{1, 0}, {3, -1}})) = 1		 c22 =  Det(({{3, 0}, {6, -1}})) = - 3		 c23 = - Det(({{3, 1}, {6, 3}})) = -3

  c31 = Det(({{1, 0}, {5, 4}})) = 4			 c32 = - Det(({{3, 0}, {2, 4}})) = -12		 c33 = Det(({{3, 1}, {2, 5}})) = 13
 Step 2: adj A is,

 (({{-1, 26, -24}, {1, -3, -3}, {4, -12, 13}}))
Step 3: Find Det[A] and divide the elements of Adj A by Det[A].  The determinant of A is, Det[A] = 3(-17) +2(1) + 6(4) = -25.  Divide all the elements  of adj A by -25 to get inverse of A.
          
         A-1 =  (({{.68, - .04, - .16}, {-1.04, .12, .48}, {.96, .12, - .52}}))
Solving through Mathematica, you need to use the command Inverse[A] as follows,
Clear[A]
A = (({{3, 1, 0}, {2, 5, 4}, {6, 3, -1}}));
MatrixForm[Inverse[A]]

(({{(17/25), -(1/25), -(4/25)}, {-(26/25), (3/25), (12/25)}, {(24/25), (3/25), -(13/25)}}))

Inverse of a 2x2 Matrix:  For a 2x2 matrix A, a short cut to find the inverse is to switch a11 with a22, change the sign of a12 and a21, and divide the elements of the new matrix (Adj A) by the determinant of A.

Example:  Find the inverse of A = (({{2, 3}, {1, 4}})).
Solution:  Switch a11 with a22, change the sign of a12 and a21, and divide all the elements by Det[A].  That is, (A^(-1)) = (1/5)(({{4, -3}, {-1, 2}})) = (({{.8, - .6}, {- .2, .4}})).
In Mathematica,
Clear[A]
A=(({{2, 3}, {1, 4}}));
MatrixForm[Inverse[A]]
(({{(4/5), -(3/5)}, {-(1/5), (2/5)}}))

Properties of Inverse Matrices:  The following are properties of inverse matrices.  Students are required to demonstrate them using numerical examples.	
1-    (A.B)-1 = B-1.A-1
2-    (A')-1 = (A-1)'
3-    Det[A-1]= (1/(Det[A]))
                  
Example: Application in Econometrics
In a regression model y = Xb + e,  where y is a nx1 vector of dependent variable, X is a nxm matrix of independent variables, b is a mx1 vector of coefficients, and e is a nx1 vector of stochastic errors, b can be estimated as (Overscript[b, ∧]) = (X'.X)-1X'.y.  For the given information on a 3x10 matrix X' = (({{2, 1, 4, 5, 12, 11, 7, 9, 5, 4}, {4, 5, 6, 2, 4, 3, 9, 11, 8, 7}, {5, 3, 8, 9, 9, 8, 5, 8, 9, 5}})) and a 1x10 vector y' = (({{8, 3, 16, 20, 66, 58, 34, 42, 25, 15}})) estimate the b coefficients.
Solution: X'.X = (({{482, 358, 458}, {358, 421, 401}, {458, 401, 519}})),  (((X' . X))^(-1)) = (({{0.0129, -0.0005, -0.0110}, {-0.0005, 0.0090, -0.0065}, {-0.0110, -0.0065, 0.0167}})), and X'.y = (({{2414}, {1694}, {2221}})).  Therefore b =  (({{0.0129, -0.0005, -0.0110}, {-0.0005, 0.0090, -0.0065}, {-0.0110, -0.0065, 0.0167}})).(({{2414}, {1694}, {2221}}))= (({{5.86}, {-0.42}, {-0.57}}))
In Mathematica, define the matrices and formulate the b as follows,
Clear[X,  y,  b]
X =  (({{2, 4, 5}, {1, 5, 3}, {4, 6, 8}, {5, 2, 9}, {12, 4, 9}, {11, 3, 8}, {7, 9, 5}, {9, 11, 8}, {5, 8, 9}, {4, 7, 5}}));	 y  =  (({{8}, {3}, {16}, {20}, {66}, {58}, {34}, {42}, {25}, {15}}));
b =N[Inverse[Transpose[X].X].(Transpose[X] . y)]
\({{5.855806278007695`}, {\(-0.41578301951624835`\)}, {\(-0.5669176965346989`\)}}\)

Product of Vectors:  Three types of vector products are inner product, outer product, and cross product of vectors.

A Mathematica Note:  You can generate vectors using either the Palettes function of the Mathematica or by listing through the curved brackets { }.  It is important to note that, vectors generated through curved brackets can be used more easily in vector operations that will follow in this chapter (commands such as Dot and Cross).  If you produce vectors through the Palettes function of the Mathematica, vectors have to be conformable for matrix or vector operations.  Otherwise Mathematica will return an error message.  For this reason, vectors are generated through the brackets in the following sections. 

Inner Product or Dot Product of Vectors: The dot product or the inner product of two vectors a = {a1, a2, a3} and b = {b1,  b2,  b3}is a scalar number found by multiplying corresponding elements of the two vectors and adding up.  Here, the dot product of a.b = a1b1 + a2b2 + a3b3.  

In Mathematica, you can use the Dot [a ,  b] command of the Mathematica to find the inner product of two vectors a and b. If you generate vectors using pallets function of the Mathematica, the two vectors should be conformable for inner product.  That is, if a and b are both a 3x1 row vectors, to find their inner product you should multiply a by b' to get a scalar number.

Example:  Total Revenue
The annual average price and the total quantity sold of a product is given as p = {({{28.50,, 32.25,, 33.15}})} and q = { ({{23400,, 22150,, 26300}})}, respectively.  Find the total revenue (TR).
Solution:  Total revenue is price time quantity or the inner product of price and quantity vectors, TR = p.q' = 28.50(23400) + 32.25(22150) + 33.15(26300) = 2,253,080.  
Using Mathematica,
Clear[p, q, TR]
p = {28.50, 32.25, 33.15};
q = {23400, 22150, 26300};
TR  =  p . q
2.2530825`*^6
Alternatively,
TR = Dot[p, q]
2.2530825`*^6
Note again that, if you are solving the problem using Mathematica, you should generate the vectors using the brackets "{ }" rather than the Pallets.  Otherwise, you can't use the Mathematica command "Dot" or multiply the vectors using "."  without making the vectors conformable for multiplication for dot product.

Outer Product of Vectors:  The outer product of two vectors a and b is a matrix which its elements are the product of the corresponding elements of the two vectors.  For example, the outer product of the 3x1 column vector a = (({{3}, {1}, {-2}})) and the 1x4 row vector b = (({{2}, {0}, {5}, {7}})) is a 3x4 matrix that its elements are the product of the corresponding elements of the two vectors a.b' = (({{3}, {1}, {-2}}))(({{2, 0, 5, 7}})) = (({{6, 0, 15, 21}, {2, 0, 5, 7}, {-4, 0, -10, -14}})).  
Using Mathematica, the outer product of the two vectors is,
Clear[a, b]
a=(({{3}, {1}, {-2}}));     b=(({{2, 0, 5, 7}}));
MatrixForm[a . b]
(({{6, 0, 15, 21}, {2, 0, 5, 7}, {-4, 0, -10, -14}}))

Cross Product of Vectors:  The cross product u × v of two vectors u and v is a vector.  The cross product is defined if the vector u and v are three-dimensional vectors or higher.  To find the cross product of two vectors u = {a1, b1, c1} and v ={a2, b2, c2}, write the two vectors in a matrix form, with the first row as a defining row with the i, j, k elements.  Select the (i^th) element,  delete the corresponding row and column, and find the determinant of remaining matrix.  This will be the first element of the cross product.  Select the (j^th) element, delete the corresponding row and column, and find the determinant of the remaining matrix.  Change the sign of this number and make it the second element of the cross product.  Continue with the process to find the third element of the cross product vector.   Note that, the sign of the elements alternate, starting with no sign change for the first element. 

Example:  Find the cross product of u = {a1,  b1,  c1} and  v = {a2,  b2,  c2}.
Solution:  Write the two vectors in a matrix form as (({{i, j, k}, {a1, b1, c1}, {a2, b2, c2}})).  Delete the column and row corresponding to the (i^th) element to get (({{b1, c1}, {b2, c2}})).  The determinant of this matrix is b1c2 - b2c1 which is the first element of the cross product u × v.  Delete the row and column corresponding to the (j^th) element to get (({{a1, c1}, {a2, c2}})).  The determinant of this matrix is a1c2 - a2c1.  The negative of this number is the second element of the cross product.  Find the third element by deleting the row and column corresponding to the (k^th) and finding the determinant. The cross product is u × v = {({{b1c2 - b2c1,, a1c2 - a2c1,, a1b2 - 
                  a2b1}})}.  Note that, the sign of the elements alternate, starting no sign change for the first element).  Using Mathematica,
Clear[u,  v]
u  =  {a1,  b1,  c1};
v  =  {a2,  b2,  c2};
MatrixForm[u×v]
(({{-b2 c1 + b1 c2}, {a2 c1 - a1 c2}, {-a2 b1 + a1 b2}}))
Note 1:  The "×" in the cross product is not the alphabet character "x" of the keyboard.  It is the "×" in the "3 BasicInput" of "Palettes" .  
Note 2:  For cross and dot multiplication, generate the vectors using curved bracket,  Don't generate it using the Palettes.  
Alternatively, you can use the Cross command
MatrixForm[Cross[u,  v]]
(({{-b2 c1 + b1 c2}, {a2 c1 - a1 c2}, {-a2 b1 + a1 b2}}))

Conversion of Vectors using direct Product:  The direct product or element by element product of two vectors produces a vector with elements which are the product of the corresponding elements of the two vectors.  For example, the product of u = {a1, b1, c1} and v = {a2, b2, c2} is a vector w = u*v = {a1a2, b1b2, c1c2}. 

Example: Find u*v for u = {2, 5, 12} and v = {3, -1, 10}
Solution: u*v = {6, -5, 120}. 
Using Mathematica, the direct product is,
Clear[u,  v]
u = {2, 5, 12};
v = {3, -1, 10};
u*v
{6, -5, 120}

Problems
1-  Given
A =   (({{1,    1}, {0, -1}}))	B = (({{2, -1}, {0, 1}}))		C = (({{2, 2}, {0, -1}}))
       a-    Find A + C' and A' - C.
       b-    Find A.B' and C'. B'.
       c-    Find A'.C\(TraditionalForm\`\^\(-1\)\)A and B.C\(TraditionalForm\`\^\(-1\)\).A.
       d-    Show that [(A.B)'.C']' = C.A.B
2-  Find the inverse of the following matrices,

       A = (({{1, 0}, {3, 2}}))        B = (({{1, 3}, {2, 10}}))
         
        C = (({{1, 3, 5}, {0, 2, 4}, {4, 2, 6}}))        D = (({{1, 0, 1}, {0, 1, 1}, {0, 0, 1}}))
        
3   Write the following system of equations
        2x1 +  x2 + 3x3 = 8
         x1 +  x2 -  x3 = 2
        3x1 - 2x2 +  x3 = 5
in the matrix form  A.X = C, where A is the matrix of constant coefficients, X' = [x1  x2  x3], and  C is the vector  of  constants C' = (({{8, 2, 5}})), and find  X = A-1.C.

II. Special Matrices: The IdentityMatrix Command         
The following special matrices will be frequently referred to and used in solving problems:

Identity Matrix:   The unit or identity matrix is a square matrix of order nxn with units in the principal diagonal and zeros everywhere else.  The following matrix is a 4x4 identity matrix.

	         I =(({{1, 0, 0, 0}, {0, 1, 0, 0}, {0, 0, 1, 0}, {0, 0, 0, 1}}))
                    
It is easy to verify that I.A = A.I = A.  

In Mathematica you can generate identity matrix using command IdentityMatrix[n], where n is the order of the identity matrix.

Example: Generate a 4x4 identity matrix.
MatrixForm[IdentityMatrix[4]]

(({{1, 0, 0, 0}, {0, 1, 0, 0}, {0, 0, 1, 0}, {0, 0, 0, 1}}))

Scalar Matrix:   Scalar matrices have a common scalar element in the principal diagonal and zeros everywhere else.  The following matrix is a 4x4 scalar matrix with a common scalar element 7.

		(({{7, 0, 0, 0}, {0, 7, 0, 0}, {0, 0, 7, 0}, {0, 0, 0, 7}}))

Scalar multiplication of a matrix A is equal to scalar matrix multiplication of that matrix. That is, kA = (kI)A = A(kI) = Ak.

Example:  Using Mathematica, create a 3x3 scalar matrix with a scalar unit 5.
MatrixForm[5 IdentityMatrix[3]]
(({{5, 0, 0}, {0, 5, 0}, {0, 0, 5}}))

Diagonal Matrix:   A diagonal matrix has scalar elements, not necessarily equal, in the principal diagonal and zeros in the off-diagonal positions.  The following matrix is an example of a 4x4 diagonal matrix.

		(({{5, 0, 0, 0}, {0, -2, 0, 0}, {0, 0, 1, 0}, {0, 0, 0, 6}}))
 
Symmetric Matrix:   If A = A', then A is called a symmetric matrix. It is obvious that A should be a square matrix.  For example, if
         
    A =(({{5, 3, 8}, {3, 7, 0}, {8, 0, 4}}))             A' = (({{5, 3, 8}, {3, 7, 0}, {8, 0, 4}}))         and A = A'

Orthogonal  Matrix:  A matrix  A  is said to be orthogonal if Det[A] ≠ 0  and A'A = I.

Example:  Show that  A =  (({{((Sqrt[2])/2), -((Sqrt[2])/2)}, {((Sqrt[2])/2), ((Sqrt[2])/2)}})) is an orthogonal matrix.
Solution:  Det[A] = 1≠ 0 and A'.A = (({{1, 0}, {0, 1}})).  Therefore, A is an orthogonal matrix.        
 Clear[A]
 A= (({{((Sqrt[2])/2), ((-(Sqrt[2]))/2)}, {((Sqrt[2])/2), ((Sqrt[2])/2)}}));
 MatrixForm[A . Transpose[A]]
 
(({{1, 0}, {0, 1}}))

Orthogonal Vectors: Two vectors u and v of the same order are called orthogonal vectors or perpendicular vectors if u'.v = 0.

Example:   Show that u = (({{2}, {-3}, {1}}))and v = (({{1}, {2}, {4}})) are orthogonal vectors.
Solution:  u'v = (({{2, -3, 1}}))(({{1}, {2}, {4}})) = 0.
Using Mathematica,
Clear[u,  v] 
u = (({{2}, {-3}, {1}}));              v = (({{1}, {2}, {4}}));

   Transpose[u] . v
{{0}}

Idempotent Matrix:  The matrix A is an idempotent matrix if  A.A = A. That is, an idempotent matrix is one that upon dot multiplication by itself remains unchanged. 

Example:  Prove that A =  (({{(2/3), (1/3), (1/3)}, {(1/3), (2/3), -(1/3)}, {(1/3), -(1/3), (2/3)}})) is an Idempotent matrix.
Solution: A.A = A, thus A is an idempotent matrix.
Using Mathematica,
MatrixForm[(({{(2/3), (1/3), (1/3)}, {(1/3), (2/3), -(1/3)}, {(1/3), -(1/3), (2/3)}})).(({{(2/3), (1/3), (1/3)}, {(1/3), (2/3), -(1/3)}, {(1/3), -(1/3), (2/3)}}))]
(({{(2/3), (1/3), (1/3)}, {(1/3), (2/3), -(1/3)}, {(1/3), -(1/3), (2/3)}}))

Idempotent Matrices and Statistics:  A very useful device for data manipulation in statistics is a vector i which contains a column of 1s.  This vector can be operated on vectors of the same order, say x, to find the sum, mean, and deviations from mean.  The sum of the elements of x is (∑_(i = 1)^n) (Subscript[x, i])  = (Subscript[x, 1])+ (Subscript[x, 2])+ . . . + (Subscript[x, n]) = i'x.  Since (∑_(i = 1)^n) (Subscript[x, i]) = n\(TraditionalForm\`\(x\&-\)\), \(TraditionalForm\`\(x\&-\)\) = (1/n)i'x.  The set of deviations from mean, then, is x - i(1/n) i'x = .  Since (1/n) = (((i' i))^(-1)),  thus x - i(1/n) i'x = x - i(((i' i))^(-1))i'x = (I - i(((i' i))^(-1))i')x, where I is a unit matrix.  Denoting (I - i(((i' i))^(-1))i') by (M^o), (M^o) = (I - i(((i' i))^(-1))i') is called the fundamental matrix in statistics and is used in transforming data to deviation from their mean.

Example: Variance
Prove that the variance of a variable x, containing n observations and defined as Var(x) = \(TraditionalForm\`∑\+\(i = 1\)\%n\((x\_i - \ \(x\&-\)\ )\)\^2\) can be written as x'(M^o)x, where  (M^o) = (I - i(((i' i))^(-1))i').
Solution:  The set of deviations from mean can be written as (M^o)x.  The variance, which is the sum square of deviations from mean, can be written in matrix as ((M^o)x)'((M^o)x) = x'(M^o)'(M^o)x.  Since (M^o) is an idempotent matrix, (M^o)'(M^o) = (M^o), and Var(x) = x'(M^o)x.

Problems
1.  Show that X.(X'.X)-1.X'  and  I - X.(X'.X)-1.X' are idempotent matrices.
2.  Show that the following matrix is an orthogonal matrix.
            
          (({{((Sqrt[3])/3), -((Sqrt[2])/2), ((Sqrt[6])/6)}, {((Sqrt[3])/3), ((Sqrt[2])/2), ((Sqrt[6])/6)}, {((Sqrt[3])/3), 0, -((Sqrt[6])/6)}}))
     
3.   Find vectors that are orthogonal to the following vectors
        a)  u' = (({{1, 0, -1}}))
        b)  u' = (({{3, -2, 5}}))
4.   Using the fundamental matrix  (M^o) = (I - i(((i' i))^(-1))i'),
	a)  Prove that (M^o) is an idempotent matrix.
	b)  Prove that sum deviations from mean of any variable equals zero.
5.   Prove that the covariance between two variables x and y, defined as Cov(x, y) = (∑_(i = 1)^n)((Subscript[x, i]) - \(TraditionalForm\`\(x\&-\)\))((Subscript[y, i]) - \(TraditionalForm\`\(y\&-\)\)) can be written as x'(M^o)y, where (M^o) =  (I - i(((i' i))^(-1))i').