This site is being phased out.

Linear operators: part 1

From Mathematics Is A Science
Jump to navigationJump to search

Since a lot of properties of linear operators have nothing to do with linearity, we start with the general setup: $f \colon X \rightarrow Y$ functions between sets.

Compositions and inverses, of functions

In calculus 1, $f \colon {\bf R} \rightarrow {\bf R}$, typically.

Definition: Given $f \colon X \rightarrow Y$, $g \colon Y \rightarrow Z$, two functions between sets $X,Y,Z$, then their composition is

$$g \circ f \colon X \rightarrow Z$$

given by

$$(g \circ f)(x) = g(f(x))$$

Note 1: Here $g \circ f$ is the name of the new function.

Note 2: Both $f$ and $g$ are input/output boxes, wired consecutively they produce the composition:

$$\stackrel{x}{\rightarrow} f \stackrel{y}{\rightarrow} \ldots \stackrel{y}{\rightarrow} g \stackrel{z}{\rightarrow}$$

Since $y \in Y$ we can wire these together.

Note 3: Observe that we read from right to left: $g(f(x))$, meaning $$x \rightarrow f(x) \rightarrow g(f(x)).$$

Definition: Given $f \colon X \rightarrow Y$, the inverse of $f$ is a function $g$ that satisfies

  • $g \circ f = {\rm id}_x$
  • $f \circ g = {\rm id}_y$

(Recall ${\rm id}_x(x)=x$ for all $x \in X$ etc)

Not all functions have inverses, but...

Theorem: The inverse is unique when it exists.

Proof: $f \colon X \rightarrow Y$, suppose $g \colon Y \rightarrow X$ and $g' \colon Y \rightarrow X$ satisfy the two equations. (Observe the analogy with inverse matrices.)

Compact proof: $$B = BI = B(AC)=(BA)C=1C=C. (*)$$

Compare: $$\begin{array} \\ {\rm function \hspace{3pt} notation} & {\rm matrix \hspace{3pt} notation} \\ {\rm compositions} & {\rm matrix \hspace{3pt} multiplication} \\ g \circ f = {\rm id}_x & BA=I \\ f \circ g = {\rm id}_y & AB=I \end{array}$$

The left column above is the definition of inverse $g$ of $f$. The right column above is the definition of inverse $B$ of $A$.

Rewrite (*) following: $B \longleftrightarrow g$, $A \longleftrightarrow f$, $C \longleftrightarrow g'$. Then

$$g(y)=g({\rm id}_Y(y)) = g(f \circ g')(y) = (g \circ f)(g'(y))={\rm id}_xg'(y) = g'(y).$$ $\blacksquare$


Notation for the inverse: $f^{-1}$.

What kind of functions have inverses?

Recall the "arrow" interpretation of function: then the inverse is the same but the arrows' directions are reversed. Except we might have a problem.

NonInjectiveFunction.png (not supposed to happen!)

This is not a function: one input -- two outputs.

Therefore we require that $f$ is one-to-one: if $x \neq x'$, then $f(x) \neq f(x')$. (same as in Calc 1)

Does every one-to-one function have the inverse? No.

Example:

NonSurjectiveFunction.png

Here we have an input without an output!

Therefore we require that $f$ is onto: if $y \in Y$, then there is $x \in X$ such that $f(x)=y$.


Theorem: $f$ has the inverse if and only if it's one-to-one and onto.

$$\begin{array}{} {\rm AKA}: & {\rm one-to-one} &= {\rm injective} \\ & {\rm onto} &= {\rm surjective} \\ & {\rm both} &= {\rm bijective} \end{array}$$


Definition: Given $f \colon X \rightarrow Y$, $f(x)$ is the image (or value) of $x \in X$. Further, $A \subset X$, then $f(A) = \{f(x) \colon x \in A \}$ is called the image of $A$ under $f$.

Then $f$ is onto if and only if $f(X)=Y$.

Definition: $f \colon X\rightarrow Y, B \subset Y$

$$f^{-1}(B) = \{x \in X \colon f(x) \in B \}$$

is called the preimage of $B$ under $f$.

Note: here "-1" does not mean the inverse.

Example: $f$ is the projection, $X = {\bf R}^2$, $Y=x$-axis, $f^{-1}(c)$ is a vertical line, $f^{-1}(B)$ is the strip:

Preimages.png.

It follows: $f$ is one-to-one if and only if $f^{-1}({\rm point})={\rm point}$.

Example: Suppose $f(x)={\rm sin \hspace{3pt}} x$.

PreimageOfSin.png

Find $f^{-1}(B)=?$, $x \in f^{-1}(B)$.

Compare these two.

  • "Given $x$, how to find $f(x)$" and "given $y$, how to find $f^{-1}(y)$":

GraphicalInterpretationOfPreimages.png

Now solve the problem:

GraphicalInterpretationOfPreimages2.png

Answer: $f^{-1}(B)$ is the union of infinitely many intervals on the $x$-axis.

Commutative diagrams

We've seen a few already.

Given a function $f \colon X \rightarrow Y$. These are possible representations: $$ \newcommand{\ra}[1]{\!\!\!\!\!\!\!\xrightarrow{\quad#1\quad}\!\!\!\!\!} \newcommand{\da}[1]{\left\downarrow{\scriptstyle#1}\vphantom{\displaystyle\int_0^1}\right.} % \begin{array}{llllllllllll} & x & \mapsto & y \\ & input & \ra{function} & output \\ & X & \ra{f} & Y \end{array} $$

The first one might come from computer science or similar and the second is the way we do it in algebra.

What if we have another function $g \colon Y \rightarrow Z$, how do we represent their composition?

The composition of $$ \newcommand{\ra}[1]{\!\!\!\!\!\!\!\xrightarrow{\quad#1\quad}\!\!\!\!\!} \newcommand{\da}[1]{\left\downarrow{\scriptstyle#1}\vphantom{\displaystyle\int_0^1}\right.} % \begin{array}{llllllllllll} X & \ra{f} & Y & \ra{g} & Z \\ \end{array} $$ can be a commutative diagram too: $$ \newcommand{\ra}[1]{\!\!\!\!\!\!\!\xrightarrow{\quad#1\quad}\!\!\!\!\!} \newcommand{\da}[1]{\left\downarrow{\scriptstyle#1}\vphantom{\displaystyle\int_0^1}\right.} % \begin{array}{llllllllllll} & X & \ra{f} & Y \\ & & \searrow ^{g \circ f}& \da{g} \\ & & & Z \end{array} $$

The fist line is just the two functions "wired together" but the second is more revealing. One can see the composition function that goes diagonally.

The point of the diagram is that you can

  • go right then down, or
  • go diagonally.

Either way, you get the same result: $$g(f(x))=(g \circ f)(x).$$

Specifically: $$ \newcommand{\ra}[1]{\!\!\!\!\!\!\!\xrightarrow{\quad#1\quad}\!\!\!\!\!} \newcommand{\da}[1]{\left\downarrow{\scriptstyle#1}\vphantom{\displaystyle\int_0^1}\right.} % \begin{array}{llllllllllll} & x \in X & \ra{f} & Y \ni f(x) \\ & & \searrow ^{g \circ f}& \da{g} \\ & (g \circ f)(x) & \in & Z \ni g(f(x)) \end{array} $$

As an example, we can use this idea to represent the inverse function, just choose $g=f^{-1}$ above:

$$ \newcommand{\ra}[1]{\!\!\!\!\!\!\!\xrightarrow{\quad#1\quad}\!\!\!\!\!} \newcommand{\da}[1]{\left\downarrow{\scriptstyle#1}\vphantom{\displaystyle\int_0^1}\right.} % \begin{array}{llllllllllll} & X & \ra{f} & Y \\ & & \searrow ^{id_X} & \da{f^{-1}} \\ & & & X \end{array} $$

More generally, the diagram may be a square.

$$ \newcommand{\ra}[1]{\!\!\!\!\!\!\!\xrightarrow{\quad#1\quad}\!\!\!\!\!} \newcommand{\da}[1]{\left\downarrow{\scriptstyle#1}\vphantom{\displaystyle\int_0^1}\right.} % \begin{array}{llllllllllll} X & \ra{f} & Y \\ \da{g'} & \searrow & \da{g} \\ X' & \ra{f'} & Y' \end{array} $$

Again, go right then down, or go down then right, same result:

$$g \circ f = f' \circ g'.$$

Both give you the diagonal function!

Then it's called a commutative diagram.

Indeed, vertical then horizontal is same as horizontal then vertical.


Back to linear operators...

Properties of linear operators

Theorem: If $A \colon X \rightarrow Y$ and $B \colon Y \rightarrow Z$ are linear operators then so is $B \circ A$.

Proof: $$\begin{array}{} (B \cdot A)(x+y) &= B(A(x+y)) & \\ &= B(A(x)+A(y)) & {\rm linearity \hspace{3pt} of \hspace{3pt}} A \\ &= B(A(x)) + B(A(y)) & {\rm linearity \hspace{3pt} of \hspace{3pt}}B \\ &= (B \circ A)(x) + (B \circ A)(y) \end{array}$$

As easy for scalar multiplication. $\blacksquare$

Theorem: The inverse of a linear operator is linear (if it exists).

Proof: Given $A \colon X \rightarrow Y$, a linear operator, and $A^{-1} \colon Y \rightarrow X$, its inverse, this is what we know:

$$ A^{-1}A = {\rm id}_X, AA^{-1}={\rm id}_Y. (*)$$

We need to show that:

$$A^{-1}(y+y')=A^{-1}(y)+A^{-1}(y').$$

for given $y,y' \in Y$. To use the linearity of $A$ on $X$:

$$ A(x+x')=A(x)+A(x') (**)$$

for given $x,x' \in X$, we match these $y,y' \in Y$ with those $x,x' \in X$.

How? Choose $x,x'$ that satisfy

$$ A(x)=y {\rm \hspace{3pt} and \hspace{3pt}} A(x')=y'. (***)$$

But do they exist? Yes, because $A$ is onto.

Consider

$$\begin{array}{} A^{-1}(y+y') &= {\rm substitute} & \\ &= A^{-1}(A(x)+A(x')) & {\rm linearity \hspace{3pt} of \hspace{3pt} }A, (**) \\ &= A^{-1}(A(x+x')) & {\rm use \hspace{3pt} definition \hspace{3pt} of \hspace{3pt} inverse}, (*) \\ &= {\rm id}_x(x+x') & {\rm use \hspace{3pt} definition \hspace{3pt} of \hspace{3pt} identity} \\ &= x+x' & \\ &= A^{-1}(y) + A^{-1}(y) & {\rm use \hspace{3pt}} (***) \end{array}$$

Same for scalar multiplication (exercise). $\blacksquare$

Linear operators and bases

We discuss the existence of linear operators, in the vector space setting $A \colon V \rightarrow U$.

We ve see these two:

  • $A(x)=rx$ for $U=V$;
  • $A(x)=0.$

We want more examples!

Example: Let' build $A \colon {\bf R}^3 \rightarrow {\bf R}^2$.

Pick any three vectors in ${\bf R}^2$ $\{u_1,u_2,u_3\}$.

Define $A \left( \left[ \begin{array}{} a_1 \\ a_2 \\ a_3 \end{array} \right] \right) = a_1u_1 + a_2u_2 + a_3u_3$.

Check linearity:

$$\begin{array}{} A \left( r\left[ \begin{array}{} a_1 \\ a_2 \\ a_3 \end{array} \right] \right) &= A \left( \left[ \begin{array}{} ra_1 \\ ra_2 \\ ra_3 \end{array} \right] \right) \\ &= a_1ru_1+a_2ru_2+a_3ru_3 \\ &= r(a_1u_1+a_2u_2+a_3u_3) \\ &= rA \left( \left[ \begin{array}{} a_1 \\ a_2 \\ a_3 \end{array} \right] \right) \end{array}.$$

Same for addition.

More generally, we have...

Theorem: Suppose $\{v_1,\ldots,v_n\}$ is a basis of $V$, arbitrary. Suppose $u_1,\ldots,u_n$ are vectors in $U$. Then there is a (unique) linear operator $A \colon V \rightarrow U$ such that $A(v_i)=u_i$, $i=1,\ldots,n$.

Furthermore, $$A \left( \sum_{i=1}^n a_iv_i \right) = \sum_{i=1}^n a_iu_i.$$

Example: Let's recall this:

$\left. \begin{array}{} {\rm dim \hspace{3pt}} {\bf P}^n &= n+1 \\ {\rm dim \hspace{3pt}} {\bf R}^{n+1} &= n+1 \end{array} \right\}$ So what?

This means that

  • there is a one-to-one onto correspondence between a basis of ${\bf P}^n$ and a basis of ${\bf R}^{n+1}$.

We can, therefore, define a function $A \colon {\bf P}^n \rightarrow {\bf R}^{n+1}$, by the theorem. It follows, then:

  • there is a one-to-one onto correspondence between the elements of the vector spaces.

And this function is linear!

Specifically, this is the correspondence between the bases for $n=2$:

  • ${\bf P}^2$, $B=\{1,x,x^2\}$ and
  • ${\bf R}^3$, $B'=\{(1,0,0),(0,1,0),(0,0,1)\}$.

Define $$\left\{ \begin{array}{} A(1) &= (1,0,0) \\ A(x) &= (0,1,0) \\ A(x^2) &= (0,0,1) \end{array} \right.$$

That, following the idea of the above theorem, defines a linear operator $A \colon {\bf P}^2 \rightarrow {\bf R}^3$.

  • $A$ is also one-to-one and onto.

Linear operators that are one-to-one and onto are called isomorphisms.

In this particular example, the images of the basis elements are also elements of a basis. In general, a function defined via the theorem does not have to be either one-to-one or onto.

Proposition: For the linear operator $A \colon V \rightarrow U$ defined in the theorem above:

Proof: For dimension 2...

(1) Suppose $A$ is not one-to-one, then there are two $x_1,x_2 \in U$ such that $A(x_1)=A(x_2)$. Suppose $$\begin{array}{} x_1 &= a_1v_1+a_2v_2+a_3v_3 \\ x_2 &= b_1v_1+b_2v_2+b_3v_3 \\ A(x_1) &= a_1e_1+a_2e_2+a_3e_3 \\ A(x_2) &= b_1e_1+b_2e_2+b_3e_3 \end{array}$$

They are equal:

$$a_1e_1+a_2e_2+a_3e_3=b_1e_1+b_2e_2+b_3e_3.$$

Collect similar terms:

$$(a_1-b_1)e_1+(a_2-b_2)e_2+(a_3-b_3)e_3=0.$$

Then the coefficients are all zero because $e_1,e_2,e_3$ are linearly independent. So $a_1=b_1$, $a_2=b_2$, $a_3=b_3$. Hence $x_1=x_2$.

(2) Exercise. $\blacksquare$

In this sense, ${\bf P}$ mirrors, mimics the algebra of ${\bf R}^3$!

Example: Compute $(x^2+x)-3(x-1)=?$. Take it to ${\bf R}^3$ under $A$, then coordinate vectors

$$\begin{array}{} (0,1,1)-3(-1,1,0) &= (0,1,1)-(-3,3,) &\in {\bf R}^3 \\ &= (3,-2,1) &\in {\bf R}^3 \\ & f^{-1} \downarrow & {\rm take \hspace{3pt} this \hspace{3pt} back \hspace{3pt} to \hspace{3pt}} {\bf P}^2 \\ & 3-2x+x^3 \in {\bf P}^2 \end{array}$$

Then ${\bf P}^2$ and ${\bf R}^3$ are indistinguishable from each other, as far as linear algebra is concerned.


Properties of compositions (mimic properties of matrices, or vice versa)

  • 1. $f \circ (g \circ h) = (f \circ g) \circ h$ (associativity)
  • 2. the inverse $f^{-1}$ is unique, if exists
  • 3. $f,g$ are one-to-one, then $f \circ g$ is one-to-one
  • 4. $f,g$ are onto, then $f \circ g$ is onto
  • 5. $(f \circ g)^{-1} = g^{-1} \circ f^{-1}$

The last one is illustrated below:

$$\stackrel{g}{\rightarrow} \stackrel{f}{\rightarrow}$$

$$\stackrel{g^{-1} }{\longleftarrow} \stackrel{f^{-1} }{\longleftarrow}$$