This site is being phased out.

Eigenvalues and eigenvectors of linear operators

From Mathematics Is A Science
Jump to navigationJump to search

Motivation

Consider a linear operator given by $A=\left[ \begin{array}{} 2 & 0 \\ 0 & 3 \end{array} \right]$, $A \colon {\bf R}^2 \rightarrow {\bf R}^2$, so $A$ is a transformation of the plane.

What is it geometrically?

Consider where $A$ takes the basis vectors:

$$\begin{array}{} e_1 = \left[ \begin{array}{} 1 \\ 0 \end{array} \right] \stackrel{A}{\rightarrow} \left[ \begin{array}{} 2 \\ 0 \end{array} \right] \\ e_2 = \left[ \begin{array}{} 0 \\ 1 \end{array} \right] \stackrel{A}{\rightarrow} \left[ \begin{array}{} 0 \\ 3 \end{array} \right] \end{array}$$

(These are columns of $A$ by the way.)

Then $A(e_1)=2e_1$, $A(e_2)=3e_2$. So $A$ stretches horizontally by 2x and vertically by 3x.

File:StretchThePlane.png

In fact, any stretch of this kind will have this form:

$$A = \left[ \begin{array}{} h & 0 \\ 0 & v \end{array} \right],$$

where $h$ horizontal coefficient, $v$ vertical coefficient.


What about a stretch but along other axes, i.e., with respect to another basis?

GeometricalInterpOfStretch.png

Given a basis $\{v_1,v_2\}$, define a linear operator with matrix $A = \left[ \begin{array}{} a & 0 \\ 0 & b \end{array} \right]$ with respect to this basis.

If this is not the standard basis, then the matrix is not diagonal.

How to find a basis that makes the matrix diagonal, if possible?

Let's analyze.

Observation: $A$ acts as scalar multiplication:

  • $A(v_1)=\lambda_1v_1$ and
  • $A(v_2)=\lambda_2v_2$,

where $\{v_1,v_2\}$ is the basis we are looking for.

Definition

Definition: Given a linear operator $A \colon V \rightarrow V$, a number $\lambda$ is called an eigenvalue of $A$ if

$$A(v)=\lambda v$$

for some non-zero vector $v \in V$. Then, $v$ is called an eigenvector of $A$ corresponding to $\lambda$.

Example: $A=I_n$, the equation to be solved:

$$\begin{array}{} Av &= \lambda v \\ I_nv=\lambda v \\ v = \lambda v \end{array}$$

So $\lambda=1$. This is the only eigenvalue.

What about eigenvectors?

All of them except $0$.

Example: $A = \left[ \begin{array}{} 2 & 0 \\ 0 & 3 \end{array} \right]$, the equation is:

$$\left[ \begin{array}{} 2 & 0 \\ 0 & 3 \end{array} \right] \left[ \begin{array}{} x \\ y \end{array} \right] = \lambda \left[ \begin{array}{} x \\ y \end{array} \right].$$

Rewrite

$$\left[ \begin{array}{} 2x \\ 3y \end{array} \right] = \left[ \begin{array}{} \lambda x \\ \lambda y \end{array} \right] \rightarrow \begin{array}{} 2x &=\lambda x \\ 3y &= \lambda y.\\ \end{array}$$

Hence

  • $x(2-\lambda )=0$, and
  • $y(3-\lambda )=0$.

Now,

  • $x \ne 0 \Rightarrow 2-\lambda =0 \Rightarrow \lambda =2$,
  • $y \ne 0 \Rightarrow 3-\lambda =0 \Rightarrow \lambda =3$.

These are the only two possibilities.

So,

$\lambda = 2$, then $y=0$, so its eigenvectors are $\left[ \begin{array}{} x \\ 0 \end{array} \right]$, $x \neq 0$.

$\lambda=3$, then $x=0$, so its eigenvectors are $\left[ \begin{array}{} 0 \\ y \end{array} \right]$, $y \neq 0$.

Except for the restriction $x \neq 0$, $y \neq 0$, eigenvectors corresponding to the eigenvalue form a subspace:

For $\lambda = 2$, $V_1 = \left\{ \left[ \begin{array}{} x \\ 0 \end{array} \right] \colon x \in {\bf R} \right\}$.

For $\lambda=3$, $V_2= \left\{\left[ \begin{array}{} 0 \\ y \end{array} \right] \colon y \in {\bf R} \right\}$.

More generally...

Eigenspaces

Definition: For a given eigenvalue $\lambda$ of $A$, the eigenspace of $A$ corresponding to $\lambda$ is

$$E_A(\lambda) = \{ v \in V \colon A(v)=\lambda v \}.$$

It's all eigenvectors plus $0$...

Theorem: An eigenspace is a subspace of $V$.

Proof: We use the Subspace Theorem, and need to show:

1. It's non-empty: $A(0)=0 = \lambda \cdot 0$.

2. It's closed under addition:

  • $u,v \in E_A \Rightarrow$
  • $A(u) = \lambda u$,
  • $A(v) = \lambda v$.

Hence $$\begin{array}{} A(u+v) &= A(u)+A(v) \\ &= \lambda u + \lambda v \\ &= \lambda(u+v), u+v \in E_a(\lambda)$. \end{array}$$

3. Similar for scalar multiplication. $\blacksquare$


Example: $E_{I_2}(1) = {\bf R}^2$.

Example: $A = \left[ \begin{array}{} 2 & 0 \\ 0 & 3 \end{array} \right]$, then

  • $E_A(2) = {\rm span}\{e_1\}$, $x$-axis.
  • $E_A(3) = {\rm span}\{e_2\}$, $y$-axis.

Example: For the zero matrix, $A=0$,

  • $Av=\lambda v$, or
  • $0 = \lambda v$.

So, $\lambda =0$, hence

  • $E_A(0)={\bf R}^2$.

Example: Projection on the $x$-axis, $P = \left[ \begin{array}{} 1 & 0 \\ 0 & 0 \end{array} \right]$,

$$ \left[ \begin{array}{} 1 & 0 \\ 0 & 0 \end{array} \right] \left[ \begin{array}{} x \\ y \end{array} \right] = \lambda \left[ \begin{array}{} x \\ y \end{array} \right], {\rm or}$$

$$\left[ \begin{array}{} x \\ 0 \end{array} \right] = \left[ \begin{array}{} \lambda x \\ \lambda y \end{array} \right], {\rm or}$$

$$x=\lambda x, 0 =\lambda y$$

so, the only possible values are $0$ and $1$.

1) $\lambda = 0 \Rightarrow $

  • $x=0 \cdot x \Rightarrow x=0$, and
  • $0=0 \cdot y \Rightarrow y$ any.

Hence $E_A(0) = \{ \left[ \begin{array}{} 0 \\ y \end{array} \right] \colon y \in {\bf R} \} = {\rm span}\{e_2\}.$

This is the $y$-axis.

AnEigenspace.png

2) $\lambda = 1 \Rightarrow $

  • $x=1 \cdot x \Rightarrow x$ any, and
  • $0=1 \cdot y \Rightarrow y=0$.

So

  • $E_A(1)={\rm span}\{e_1\}$.

This is the $x$-axis.


Example: $A$ is the $90^o$ rotation:

$A = \left[ \begin{array}{} 0 & -1 \\ 1 & 0 \end{array} \right]$.

Then

$$\left[ \begin{array}{} 0 & -1 \\ 1 & 0 \end{array} \right] \left[ \begin{array}{} x \\ y \end{array} \right] = \lambda \left[ \begin{array}{} x \\ y \end{array} \right],$$

or

$$\left\{ \begin{array}{} -y &= \lambda x \\ x = \lambda y \end{array} \right.$$

Solve it.

$$\left\{ \begin{array}{} -xy &= \lambda x^2 \\ xy &= \lambda y^2 \end{array} \right.,$$

so $\lambda x^2 = -\lambda y^2$.

Two cases:

  1. $\lambda = 0$, then $x=y=0$; and
  2. $\lambda \neq 0$, then $x^2=-y^2$.

At least one would have to be imaginary.

Since we are talking about $A \colon {\bf R}^2 \rightarrow {\bf R}^2$, $x,y$ have to be real...

Alternative formula

Shortcut: $\lambda$ is an eigenvalue for $A$ (basis fixed). For some $v \in {\bf R}^n \setminus \{0\}$,

$$\begin{array}{} {\rm definition} &\Longleftrightarrow A(v)=\lambda v \\ &\Longleftrightarrow A(v)-\lambda v = 0 \\ {\rm A \hspace{3pt} is \hspace{3pt} a \hspace{3pt} matrix} &\Longleftrightarrow Av-\lambda v=0 \\ &\Longleftrightarrow (A-\lambda)v=0 ({\rm \hspace{3pt} but \hspace{3pt}} \lambda {\rm \hspace{3pt} is \hspace{3pt} a \hspace{3pt} number!}) \\ &{\rm better: \hspace{3pt}} (A- \Lambda)v=0, \\ \Lambda=\left[ \begin{array}{} \lambda & \ldots & 0 & 0 \\ 0 & \lambda & \ldots & 0 \\ \vdots \\ 0 & \ldots & 0 & \lambda \end{array} \right] = \lambda I_n \\ &{\rm or \hspace{3pt}} (A-\lambda I_n)v=0 {\rm \hspace{3pt} for \hspace{3pt} some \hspace{3pt}} v \in {\bf R}^n \setminus \{0\}. \end{array}$$

This is a matrix equation

  • matrix $A - \lambda I_n$
  • $v$ is the unknown
  • right hand side is $0$

Recall theorem: there is no non-zero solution if and only if rank of the matrix $=n$.

So this is an isomorphism and the matrix is invertible...

Use the theorem: $\det (A-\lambda I_n) \neq 0$.

Then, a non-zero solution exists if and only if ${\rm det}(A-\lambda I_n)=0$.

Theorem: Given a linear operator $A \colon {\bf R}^n \rightarrow {\bf R}^n$, bases fixed. Then $\lambda$ is an eigenvector if and only if $\det (A-\lambda I_n)=0$.


Example: $A = \left[ \begin{array}{} 0 & -1 \\ 1 & 0 \end{array} \right]$.

$$\det \left( \left[ \begin{array}{} 0 & -1 \\ 1 & 0 \end{array} - \lambda \left[ \begin{array}{} 1 & 0 \\ 0 & 1 \end{array} \right] \right] \right) = \det \left[ \begin{array}{} -\lambda & -1 \\ 1 & -\lambda \end{array} \right] = \lambda^2 + 1 = 0$$

Solve, then $\lambda = \pm i$.

Try $Av=\lambda v$, for $x, y \in {\bf R}$,

$$\left[ \begin{array}{} 0 & -1 \\ 1 & 0 \end{array} \right] \left[ \begin{array}{} x \\ y \end{array} \right] = i \left[ \begin{array}{} x \\ y \end{array} \right]$$

$$-y = ix,$$

$$x = iy,$$ $x,y$ real.


Note: "Eigen" value = characteristic value.

The characteristic polynomial of matrix $A$ is

$$\chi_A(\lambda) = \det(A - \lambda I_n).$$

This is a polynomial of $\lambda$!

Example: $\lambda^2+1$ for above.

Example: $A = \left[ \begin{array}{} 2 & 0 \\ 0 & 3 \end{array} \right]$, $$\chi_A(\lambda) = \det \left[ \begin{array}{} 2-\lambda & 0 \\ 0 & 3-\lambda \end{array} \right] = (2-\lambda)(3-\lambda) = 0$$

Solve, so $\lambda=2,3$.

Example: $A=\left[ \begin{array}{} 1 & 0 \\ 0 & 0 \end{array} \right]$, solve it. So $\lambda=1,0$.

Properties

  • Fact 1: If $A$ is $n \times n$, then the degree of $\chi_A$ is $n$.
  • Fact 2: Eigenvalues are the roots of the characteristic polynomial.
  • Fact 3: Number of roots is $n$, counting multiplicities (including complex roots).


Theorem: $A$ is diagonalizable if all roots of $\chi_A$ are real with multiplicity $1$.

Why?

Find a basis for ${\bf R}^n$ that consists of the eigenvectors corresponding to these eigenvalues.

Suppose $P$ is the transition matrix for these bases. Then $D=P^{-1}AP$ is a diagonal matrix.

In this case, $D$ and $A$ are called similar, $D \sim A$.

Theorem: Similarity is an equivalence relation on ${\bf M}(n,n)$.

Proof: The axioms:

1. $A \sim A$, just choose $P=I_n$.

2. $A \sim B \longleftrightarrow A = P^{-1}BP$, how?

  • $B \sim A \longleftrightarrow B=Q^{-1}AQ$.

These are equivalent to:

  • $A = P^{-1}BP \longleftrightarrow B=Q^{-1}AQ.$

Find $Q$.

Choose: $Q = P^{-1}$.

$$\begin{array}{} B &= (P^{-1})^{-1}AP^{-1} = PAP^{-1} \\ P^{-1}B &= AP^{-1} \\ P^{-1}BP &= A \\ P^{-1}BP = A, \end{array}$$

as required.

3. $A \sim B \sim C \rightarrow A \sim C$, how?

We need this

  • $C = P^{-1}BP,$
  • $B=Q^{-1}AQ,$

to imply that

  • $C=R^{-1}AR$.

We need to find $R$.

Substitute:

$$\begin{array}{} C &= P^{-1}(Q^{-1}AQ)P \\ &= P^{-1}Q^{-1}AQP \end{array}$$ Compare to $C$ above...

Choose $R=QP$, finish (exercise). $\blacksquare$


Theorem: The characteristic polynomial is preserved under similarity:

$$\chi_{P^{-1}AP} = \chi_A.$$

Proof: Recall: $$\chi_{P^{-1}AP}(\lambda) = \det (\lambda I - P^{-1}AP).$$ We need to get rid of $P$ here.

Idea: distribute $P$, then take $P$ out, use this

Fact: the determinant is preserved under similarity: $$\det (P^{-1}AP)=\det A.$$ Then $$\chi_{P^{-1}AP}(\lambda) =$$ $$={\rm det}(\lambda P^{-1}IP - P^{-1}AP)$$ factor $P^{-1}$ and $P$ out... $$={\rm det}(P^{-1} \lambda IP - P^{-1}AP)$$ use similarity... $$={\rm det}[P^{-1} (\lambda I - A)P]$$ or use the multiplicative property of determinants instead... $$={\rm det}(\lambda I - A)$$ $$=\chi_A(\lambda).$$ $\blacksquare$

Another characteristic of matrices preserved under similarity is the trace:

$${\rm tr \hspace{3pt}}A = \displaystyle\sum_{k=1}^n a_{k_k},$$

the sum of the diagonal elements of $A$.

Theorem: The trace is linear operator: ${\rm tr \hspace{3pt}} \colon {\bf M}(n,n) \rightarrow {\bf R}$. (just like the determinant)

Proof: $$\begin{array}{} {\rm tr}(A+B) &= \sum_{k=1}^n (a_{k_k} + b_{k_k}) \\ &= \sum_{k=1}^n a_{k_k} + \sum_{k=1}^n b_{k_k} \\ &= {\rm tr \hspace{3pt}}A + {\rm tr \hspace{3pt}}B. \end{array}$$

Same, ${\rm tr}(rA)$. $\blacksquare$


Theorem: ${\rm tr}(AB) = {\rm tr}(BA)$.

Proof: $C=AB$, $C^1 = BA$. Recall

$$c_{pq} = \sum_{i=1}^n a_{pi} b_{iq}.$$

Then

$${\rm tr}(AB) = \sum_{k=1}^n c_{k_k} = \sum_{k=1}^n \sum_{i=1}^n a_{ki} + b_{ik}$$

$${\rm tr}(BA) = \sum_{k=1}^n c^1_{k_k} = \sum_{k=1}^n \sum_{i=1}^n b_{ki}a_{ik} = \sum_{k=1}^n \sum_{i=1}^n a_{ik}b_{ki}$$

These two sums don't seem to match.

What do we do? Compare them closely:

$${\rm tr}(AB) = \sum_{ki} a_{ki} b_{ik}$$

$${\rm tr}(BA) = \sum_{ki} a_{ik} b_{ki}$$

They are equal, why?

Recall that changing (as in $\displaystyle\int_0^1 x^2 dx = \displaystyle\int_0^1 t^2 dt$) names of variables is ok, when the outcome is independent of its.

Rename $k$ and $i$. How about $k \rightarrow i$ and $i \rightarrow k$.

$${\rm tr}(BA) = \sum_{ik}a_{ki}b_{ik} = {\rm tr}(AB).$$ $\blacksquare$

The idea comes from: sum of all entries in a table, rows first or columns first, same.

Theorem: The trace is preserved under similarity:

$${\rm tr}(P^{-1}AP) = {\rm tr}(A).$$

Proof: Use the last theorem

$${\rm tr}(P^{-1}AP) = {\rm tr}(PP^{-1}A) = {\rm tr}(IQ) = {\rm tr}(A).$$

$\blacksquare$

Exercise: Find all matrices similar to $I$.