Introduction to Diagonalization, Eigenvalues, and Eigenvectors [Linear Algebra I Studied #17]

Picking right up where we left off.

Remember the question we were left with? “Ugh!! How on earth do I find $R$?!”

Like, how on earth are you supposed to go hunt down the right basis~~! Yeah yeah yeah yeah, apparently finding that basis is genuinely brutal.

BUT — apparently you can still diagonalize even if you don’t know the right basis to begin with!! Let’s go figure out how!! Here’s how we’ll figure it out: we’ll start from a matrix that’s already been diagonalized and follow the clues from there.

We weren’t trying to diagonalize anything back then, but in the change-of-basis part, when we did a change of basis, it turned out (looking at it now) that we’d diagonalized something.

That equation I wrote like that — I want to say it a different way now.

Multiplying both sides by R: RA′ = AR, that is (1, −1; 1, 1)(4, 0; 0, 6) = (5, −1; −1, 5)(1, −1; 1, 1)

Here’s how I want to read this result. Rip it apart: the column vector of column 1 and the column vector of column 2, separately.

Column by column: column 1 gives 4(1, 1) = A(1, 1), column 2 gives 6(−1, 1) = A(−1, 1)

Wait wait WAIT?! We multiplied our original matrix $A$ by some vector, and it came out as that same vector times a scalar?!

This property… if we can find 2 vectors with this property for matrix $A$ (pointing in two different directions), AND find that ‘scalar’!! — boom, we can diagonalize it!!

Let’s start with a $2 \times 2$ matrix, A = (a, b; c, d) .

For a matrix $A$ like this, we want to find 2 vectors with that property. Call those vectors X₁, X₂ , and call ’that scalar’ for each of them λ₁, λ₂ . And I’ll write it like this.

Both relations: λ₁X₁ = AX₁ and λ₂X₂ = AX₂

That holds, right~ One expression that covers both of them, λX = AX — let’s go with that and keep moving.

From λX = AX to λIX = AX and then (A − λI)X = 0, written out with the 2 × 2 matrix A − λI

Whoa!! That’s a homogeneous equation!! And here’s the deal — if that matrix is invertible, the only solution is the trivial solution…

And there’s a reason we have to stop $X$ from being the zero vector (the trivial solution). If $X = 0$, that means the matrix $R$ is non-invertible, which means $A' = R^{-1}AR$ doesn’t even hold!! We have to block the situation where $X$ has nothing but the trivial solution!!

In other words, the matrix A − λI = (a − λ, b; c, d − λ) has gotta be non-invertible!! So,

The condition det(A − λI) = 0, written out as det(a − λ, b; c, d − λ) = 0

has to hold!!

Expanding the determinant: (a − λ)(d − λ) − bc = 0, that is λ² − (a + d)λ + (ad − bc) = 0

A quadratic equation. Hmmm~~ a quadratic you can crack with the quadratic formula, right?? That way we can find the two values of $\lambda$ that satisfy the equation, λ₁, λ₂ .

For this matrix, A = (5, −1; −1, 5) , shall we go find λ₁, λ₂ ?!

For A = (5, −1; −1, 5): det(A − λI) = (5 − λ)² − 1 = 0, so 5 − λ = ±1 and λ = 6 or 4

Now I want to find the X₁, X₂ that make this hold, one for each!! For the λ₁, λ₂ we found, I mean. Plug each one into

The equation λX = AX, written as λIX = AX

and find the $X$ that satisfies it — that’s how we’ll get them.

For λ = 4: (4 − 5, 1; 1, 4 − 5)X₁ = 0, that is (−1, 1; 1, −1)X₁ = 0

The $X_1$ that satisfies this — if you think about it for a sec,

there are a TON of them!! Same deal for $X_2$,

a whole bunch, like this.

OK so, we get an $X_1$ and an $X_2$, and if we line the two up side by side, (1, −1; 1, 1) = R , we call that $R$, and we can find $A'$.

Eigenvalues and eigenvectors

If we run the same play on an $n \times n$ matrix, we need to dig up $n$ vectors and their scalar values that satisfy λᵢXᵢ = AXᵢ like this.

(I’m getting tired of saying “vector” and “scalar” over and over — the nonzero vectors and the scalars that satisfy this kind of equation are called eigenvectors and eigenvalues, apparently.) They’re called that because once a matrix $A$ is pinned down — bam!! — they come right along as $A$’s very own (’eigen’ means ‘own’).

So, like we just learned, to find the $\lambda$ and $X$ that satisfy λᵢXᵢ = AXᵢ , we write it in matrix form, (A − λI)X = 0 , and to block the trivial-solution-only situation, det(A − λᵢI) = 0 absolutely has to hold — we know that now.

Then we get an equation of degree $n$ in $\lambda$. For each root, we look for eigenvectors. If we can choose $n$ linearly independent eigenvectors over the field we’re working in, we build $R$ from them, and $A'=R^{-1}AR$ is diagonal. Repeated roots don’t automatically give us enough independent vectors.

What $A'$ will look like —

A′ is the diagonal matrix with λ₁, λ₂, …, λₙ down the diagonal and zeros everywhere else

— we can guess from the example above!!

(It doesn’t necessarily turn out like this!! Meaning — there’s an assumption I’ve been quietly making on my own without telling you. Starting from the next post I’ll bring that assumption out little by little, and from then on we go case by case.)

Why diagonal matrices are nice

And to wrap up this post, let me introduce a few reasons diagonal matrices are nice.

① You can see the geometric meaning super easily!!

When you multiply the matrix A = (5, −1; −1, 5) by the eigenvectors (1, 1) & (−1, 1) , what comes out is, respectively, 4(1, 1), 6(−1, 1) — and what this means is that the linear map $A$ is doing this:

Sketch: the eigenvectors X₁ = (1, 1) and X₂ = (−1, 1) in black on the axes, and the red dotted arrows 4X₁ and 6X₂ along the same two directions

multiply the vector $X_1$ by the matrix $A$ and it becomes 4 times longer in that direction; multiply the vector $X_2$ by the matrix $A$ and it becomes 6 times longer in that direction. You can see that geometric meaning of the matrix super easily!!

And using this, when you multiply any vector by the matrix $A$, you can also easily tell what vector comes out!! For example, split v = (0, 2) using the $X_1$, $X_2$ we found.

Then what vector turns into is easy to see~~ that’s what I’m saying heh heh heh.

② (powers) become a piece of cake!!

A matrix is fundamentally a linear map — it has the property of turning a vector into a different vector. When we want to know , doing that turning over and over in a row, if we use , then:

Powers: A·····A = Aⁿ is a pain to brute-force; A′ⁿ = (R⁻¹AR)·····(R⁻¹AR) = R⁻¹AⁿR, so Aⁿ = RA′ⁿR⁻¹, and A′ is diagonal, so its powers are a breeze

③ Plugging a matrix into a function written as a Taylor expansion.

OK so what I mean is — written as a Taylor expansion becomes

Taylor series: eˣ = lim (1 + 1/n)ⁿˣ = lim (1 + x/n)ⁿ = Σ xⁿ/n! = 1 + x + x²/2! + ⋯

and if you put the matrix $A$ into this function,

Matrix exponential: e^A = I + A + A²/2! + ⋯ = Σ Aⁿ/n!

it shows up in power form like this, so doing it like ② makes it easy again heh heh heh. Anyway, for handling the powers that pop up in a Taylor expansion, diagonal matrices are handy!!

④ Systems of differential equations become easy to solve!!

When an equation shows up, the main goal of whoever’s looking at it is “find the unknown that satisfies it.” But when a differential equation shows up, the main goal of whoever’s facing it is “find the function that satisfies it.” So with a system of differential equations, the goal is finding the functions that satisfy everything ‘all at once’!!

Example: f′(x) = 5f(x) − g(x) and g′(x) = −f(x) + 5g(x)

Let me put this in matrix form.

The same system in matrix form: d/dx (f, g) = (5, −1; −1, 5)(f, g), that is dX/dx = AX

So now we need to find the $X$ that satisfies this?!

And here — if we use a diagonal matrix!!

Decoupling: d/dx R⁻¹X = R⁻¹ARR⁻¹X, and with R⁻¹X = Y = (h(x), k(x)) it becomes d/dx (h, k) = (4, 0; 0, 6)(h, k)

The instant it’s written with a diagonal matrix, the tangle between the functions vanishes, and it turns into a differential equation you can actually solve.

How to actually solve the differential equations — obviously I’m skipping that here. This is linear algebra time, after all. Check this out instead: Introduction to the Differential Equations Special Series (Differential Equations Special #1)

Solutions: h(x) = r e^(4x) and k(x) = μ e^(6x)

Solved, just like that!! That’s why diagonal matrices are useful.

And like I said earlier — from here on out we go case by case!! Everything above is honestly just a simple intro to diagonalization. The real start of diagonalization is the very next post.

Heads-up. The figures here are the original notes, slips and all. A few things to watch for:

In the pictures that write out $A - \lambda I$ for $A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$, the bottom-right entry should read $d - \lambda$, not $c - \lambda$ (the determinant right after uses $d - \lambda$).
In the worked example, $A - \lambda I$ for $A = \begin{pmatrix} 5 & -1 \\ -1 & 5 \end{pmatrix}$ is $\begin{pmatrix} 5 - \lambda & -1 \\ -1 & 5 - \lambda \end{pmatrix}$; the picture writes the off-diagonal entries as 1, which gives the same determinant $(5 - \lambda)^2 - 1$, so $\lambda = 6$ or $4$ stands.
The diagonal matrix with $\lambda_1, \lambda_2, \ldots, \lambda_n$ on the diagonal is $A' = R^{-1}AR$; the picture labels it $A$.

Eigenvalues and eigenvectors

Why diagonal matrices are nice

Comments