This site is being phased out.

Matrices: part 1

From Mathematics Is A Science
Jump to navigationJump to search

Where do matrices come from?

This is how matrices are typically appear for the first time (in precalculus, via mixtures).

Problem 1: Given: Kenyan coffee - $\$3$ per pound, Columbian coffee - $\$5$ per pound. How much of each do you need to have $10$ pounds of blend with $\$3.50$ per pound?

Easy: Let $x$ be the weight of the Kenyan coffee and let $y$ be the weight of Columbian coffee.

Then $$\begin{array}{} x+y &= 10 \\ 3x+5y &= 3.50 \end{array}$$

Solve by substitution or by using the "augmented matrix":

$$\left[ \begin{array}{cc|c} 1 & 1 & 10 \\ 3 & 5 & 35 \end{array} \right] = \left[ \begin{array}{c|c} A & B \end{array} \right]$$

Done.

Simple, but still looks more complicated than necessary. Let's go down to dimension 1.

Problem 0: Suppose there is only Kenyan coffee, $\$3$ per pound. How much do we get for $\$60$? In other words, we want to make a "blend" one one ingredient.

Solution:

$$3x=60 \rightarrow x=\frac{60}{3}.$$

The end.

What's the advantage over the solution of Problem 1?

There is a single equation (a linear one).

We want to mimic this (scalar) equation, to get an equation for the system of equations in Problem 1.


Here's how we can handle this. Let's align these:

  • $ax=b \rightarrow x=\frac{b}{a}$, ($a \neq 0$).

We want then for Problem 1:

  • $AX=B \rightarrow X = \frac{B}{A}$?

But how?

We put the data in tables (matrices) first: $$A = \left[ \begin{array}{} 1 & 1 \\ 3 & 5 \end{array} \right], B = \left[ \begin{array}{} 10 \\ 35 \end{array} \right], X = \left[ \begin{array}{} x \\ y \end{array} \right]$$

  • The first matrix is $2 \times 2$: the coefficients of $x,y$.
  • The second one is $2 \times 1$: the right hand side, "free" terms.
  • The third one is $2 \times 1$: the unknowns.

Then $AX=B$ is called a matrix equation and it's supposed to capture the system of equations in Problem 1.

One thing remains: What is the meaning of $AX$?

Matrix multiplication

Compare the original system of equations to $AX=B$.

$$\begin{array}{} x+y &= 10 \\ 3x+5y &=35 \end{array} \longleftrightarrow \left[ \begin{array}{} 1 & 1 \\ 3 & 5 \end{array} \right] \left[ \begin{array}{} x \\ y \end{array} \right] = \left[ \begin{array}{} 10 \\ 35 \end{array} \right]$$

We can see these equations in the matrices.

Equations in matrices.png

First:

$$\left[ \begin{array}{} 1 & 1 \end{array} \right] \left[ \begin{array}{} x \\ y \end{array} \right] = \left[ \begin{array}{} 10 \end{array} \right]$$

should be $1 \cdot x + 1 \cdot y = 10$, the first entry in the right hand side.

Second:

$$\left[ \begin{array}{} 3 & 5 \end{array} \right] \left[ \begin{array}{} x \\ y \end{array} \right] = \left[ \begin{array}{} 35 \end{array} \right]$$

should be $3x+5y=35$, the second entry in the right hand side.

This suggests what the meaning of $AX$ should be. We "multiply" rows in $A$ by column(s) in $X$!

To confirm, compute:

$$\left[ \begin{array}{} 1 & 2 \\ 3 & 4 \end{array} \right] \left[ \begin{array}{} 2 \\ 3 \end{array} \right] = \left[ \begin{array}{} 1 \cdot 2 + 2 \cdot 3 \\ 3 \cdot 2 + 4 \cdot 3 \end{array} \right] = \left[ \begin{array}{} 8 \\ 18 \end{array} \right]$$

On the left hand side we have a $2 \times 2$ multiplied with a $2 \times 1$, the inner numbers match!

Since length row $=$ length of column, we can carry out this operation.

Idea: Match a row of $A$ and a column of $B$, pairwise multiply, then add the results. The sum is a single entry in $AB$!

MultiplyingMatrices.png


This is the simplest case:

$$\left[ \begin{array}{} 1 & 2 & 0 & 1 & -1 \end{array} \right] \left[ \begin{array}{} 1 \\ 0 \\ 2 \\ 3 \\ 2 \end{array} \right] = \left[ \begin{array}{} 2 \end{array} \right]$$

In the above, we have a $1 \times 5$ multiplied with a $5 \times 1$. We think of the $5$'s as "cancelling" to yield a $1 \times 1$ matrix.

Let's see where $2$ comes from. Here's the algebra: $$\begin{array}{} & 1 & 2 & 0 & 1 & -1 \\ \times & 1 & 0 & 2 & 3 & 2 \\ \hline \\ & 1 +& 0 +& 0 +& 3 -& 2 =2 \end{array}$$ We multiply vertically then add the results horizontally.

Note: the product of a $1 \times 5$ matrix and a $5 \times 1$ matrix. Think of these as vectors, then the result is called the inner product (discussed later).

Example: Even though these are the same two matrices, this is on the opposite end of the spectrum (the columns and rows are very short):

Matrix multiplication 1.png

$$\left[ \begin{array}{} 1 \\ 0 \\ 2 \\ 3 \\ 2 \end{array} \right] \left[ \begin{array}{} 1 & 2 & 0 & 1 & -1 \end{array} \right] = \left[ \begin{array}{} 1 & 2 & 0 & 1 & -1 \\ 0 & 0 & 0 & 0 & 1 \\ 2 & 4 & 0 & 2 & -2 \\ \vdots \end{array} \right]$$

Here the left hand side is $5 \times 1$ multiplied with a $1 \times 5$ yielding a $5 \times 5$. It's very much like the multiplication table.

Example:

$$\left[ \begin{array}{} 1 & 2 & 0 \\ 3 & 4 & 1 \end{array} \right] \left[ \begin{array}{} 1 & 2 \\ 0 & 1 \\ 1 & 1 \end{array} \right] = \left[ \begin{array}{} 1\cdot 1 + 2 \cdot 0 + 0\cdot 1 & \ldots \\ \vdots \end{array} \right]$$

Here we have a $2 \times 3$ multiplied with a $3 \times 2$ to yield a $2 \times 2$.


More about matrices

So, an $m \times n$ matrix is a table of real numbers with $m$ rows and $n$ columns:

RowsAndColumns.png.

Notation: $A = \{a_{ij}\}$ (or $[a_{ij}]$). Here $a_{ij}$ represents the position in the matrix:

RowsAndColumns2.png

Where is $a_{2,1}$? It's the entry at the $2^{\rm nd}$ row, $1^{\rm st}$ column.

$$a_{21}=3 \longleftrightarrow A = \left[ \begin{array}{} (*) & (*) \\ 3 & (*) \\ (*) & (*) \end{array} \right]$$

One can also think of $a_{ij}$ as a function of two variables, when the entries are give by a formula.

Example: $A = \{i+j\}$, $3 \times 3$, what is it?

$a_{ij} = i+j$. To find the entries plug in $i=1,2,3$ and $j=1,2,3$.

$$\begin{array}{} a_{11} &= 1 + 1 &= 2 \\ a_{12} &= 1+2 &= 3 \\ a_{13} &= 1+3 &= 4 \\ {\rm etc} \end{array}$$

Now form a matrix:

$$\left[ \begin{array}{c|ccc} i \setminus j & 1 & 2 & 3 \\ \hline \\ 1 & 2 & 3 & 4 \\ 2 & 3 & 4 & 5 \\ 3 & 4 & 5 & 6 \end{array} \right]$$


What's the difference from a function? Compare $a_{ij}$ with $f(x,y)=x+y$.

First, $i,j$ are positive integers, while $x,y$ are real.

Second the function is defined on the $(x,y)$-plane:

ExampleOfFunction.png

For the matrix it looks the same, but, in fact, it's transposed and flipped:

ExampleOfMatrix.png

Algebra of matrices

Recall, that matrices -- of given, fixed dimensions -- form a vector space with respect to addition and scalar multiplication.

In ${\bf M}(m,n)$, we have

  • 1. if $A = \{a_{ij}\}$ and $B=\{b_{ij}\}$, then $$A + B = \{a_{ij}+b_{ij}\}.$$

AddingMatrices.png

  • 2. if $A = \{a_{ij}\}$ and $t \in {\bf R}$, then

$$rA = \{ra_{ij}\}.$$

Now, this new operation, matrix multiplication, may be outside this vector space. Just look at the dimensions of the product (unless it's $n \times n$)

DimensionsOfMatrixProduct.png

Let's concentrate on a single entry in the product $C=\{c_{pq}\}$ of matrices $A=\{a_{ij}\},B=\{b_{ij}\}$.

Observe, $c_{pq}$ is the "inner product" of $p^{\rm th}$ row of $A$ and $q^{\rm th}$ column of $B$.

Consider:

  • $p^{\rm th}$ row of $A$ is ($1^{\rm st}$ index $p$)

$\left[ \begin{array}{} a_{p1} & a_{p2} & \ldots & a_{pn} \end{array} \right]$

  • $q^{\rm th}$ column of $B$ is ($2^{\rm nd}$ index, $q$)

$\left[ \begin{array}{} b_{1q} \\ b_{2q} \\ \vdots \\ b_{nq} \end{array} \right]$

So, $$c_{pq} = a_{p1}b_{1q} + a_{p2}b_{2q} + \ldots + a_{pn}b_{nq}.$$

Now we can write the whole thing, the matrix product of an $n \times m$ matrix $\{a_{ij}\}$ and an $m \times k$ matrix $\{b_{ij}\}$ is $C = \{c_{pq}\}$, an $n \times k$ matrix with $$c_{pq} = \sum_{i=1}^n a_{pi}b_{iq}.$$

What about the vector space, ${\bf M}(n,m)$? When can we both add and multiply?

Given $m \times n$ matrix $A$ and $p \times q$ matrix $B$,

  • 1. $A+B$ makes sense only if $m=p$ and $n=q$.
  • 2. $AB$ makes sense only if $n=p$.

Both make sense only if both $A,B$ are $n \times n$. These are called square matrices.

To summarize, the set of square matrices ${\bf M}(n,n)$ has addition, scalar multiplication, and matrix multiplication.

Observe, scalar multiplication can be replaced with matrix multiplication:

ReplaceMatrixMultWithScalarMult.png