The First Decomposition Theorem

Starting with this post, I’m going to dig into blocking and diagonalization in more detail. We’ll use the Decomposition Theorem to look at them more, more, more, more closely. That’s the whole reason we went and learned polynomials, with the integers as our toolkit.

So — that matrix from last time:

I want to describe this guy in slightly different language, and in more detail. Going through the diagonalization, we can get the characteristic polynomial .

The roots of this characteristic polynomial are 4 and 6, so we know the eigenvalues of the matrix are 4 and 6, and calling them , we find, one for each, the vectors that satisfy the equations

namely .

We found them one by one like this and called them eigenvectors!!!! And then we stack neatly side by side to build

like this, so we get a matrix like

and by hitting A with $R^{-1}$ and $R$ on both sides, bam bam, we managed to diagonalize A. The whole principle

is packed in here.

So then: acting on A like this, , we made A’. And the upshot was that A’ is just A expressed in a different basis… that was the whole story from waaaay back.

OK, let’s go back over what those formulas actually mean.

Look at this formula. The vector $X_1$ is a solution of this equation, and since the right-hand side is zero (it’s a homogeneous equation), we can also say $X_1$ belongs to the kernel of the matrix .

Same deal for $X_2$.

The dimension of each of these kernels was 1 (I’ll skip the proof here). So $X_1$ and $X_2$ each span their entire respective kernel all by themselves.

And so, taking as a basis for the 2-dimensional space, we built a new matrix

like this!!!! (The linear map represented by A takes the form A’ when written in a different basis~)

So if you think hard about that formula,

it’s also saying that the 2-dimensional space splits into two separate 1-dimensional spaces.

Splitting it like this, a vector that lives inside stays inside no matter how many times you multiply it by A, and the same goes for the subscript-2 one. You can see this kind of thing in diagonalization or blocking in general.

Which is to say, diagonalization and blocking can be described as ‘decomposing’ some vector space…

Not sure what that means exactly, but it’s something to do with ‘decomposition.’

More general cases

OK, now let’s look at diagonalization more generally, in hairier cases.

Suppose we have an $n \times n$ matrix like this, and its characteristic polynomial… ※ Careful, assumption ※ Assume the characteristic polynomial factors completely into first-degree factors, all different.

In this case, we said before that ‘diagonalization always works.’ Hmm OK then,

we’ll find $n$ vectors that satisfy this. We already know each satisfies , right?!

And (since we said diagonalization works) we go grab all $n$ of the and bundle them up into a matrix R. Well… R is invertible, of course (we proved that earlier, too), so that means the are $n$ linearly independent vectors, which means they form a basis!!!!

So if you’ve got any vector Y in the $n$-dimensional space, it can be written as a linear combination of the $n$ , so we can write this:

Which is to say, the $n$-dimensional space decomposes into the $n$ pieces !!

The reason we deliberately use the direct sum symbol here is to stress that the vectors of the space can be built as linear combinations of vectors in those pieces, and that the way of doing it is unique. What I mean by that:

here the way of writing $(x, y, z)$ is not unique, right?! That’s why this one is written without the direct sum, with just a plain old +. That said, it’s not like you have to use the direct sum every time the way is unique, but apparently it’s better to use it when you want to stress uniqueness.

Anyway, so we call the diagonal matrix A’, and then we can write the eigenvectors again,

in the new basis!!!!!

When there’s multiplicity

OK now let’s think about the case where the characteristic polynomial doesn’t factor into all-different first-degree terms; instead it has ‘multiplicity.’

In this case, if A can still be diagonalized, then going by the principle from before, A’ would look like this:

It’s got this structure where small square diagonal matrices gather to form one big diagonal matrix!!! — you could say!! If we think of splitting this $n$-dimensional space up by the eigenvalues of those diagonal matrices, splitting everything by them:

and here, , I mean,

can be written as a span like this. Hmmm~ you may have forgotten this part.

Each eigenvalue has to bring along as many linearly independent eigenvectors as its own multiplicity (that’s what lets A be diagonalized)!!! So is also called an eigenspace, apparently. The dimension of each one would be its multiplicity: for the first one, and so on.

Let me sum this up one more time, with a slightly different reading.

In this case the characteristic polynomial is

like this, and when we diagonalize, the $n$-dimensional space gets split into

$n$ spaces like that.

cf.) By the Cayley–Hamilton theorem, what happens if we plug A itself in for $t$?!

It became zero, right.

We got this, didn’t we.

Wait, but the matrices in those kernels and the matrices we’re multiplying in Cayley–Hamilton are the same thing!!! To say it again: if there’s a polynomial that becomes 0 when you plug A in for $t$, that polynomial is showing you how the $n$-dimensional space decomposes.

Can’t we say that?????

We’ll come back to this later.

For this case, let’s now write the polynomial

like this. Since it’s the characteristic polynomial with all the multiplicities stripped off, it’s a different polynomial, and that’s why I wrote it as $f(t)$.

Going back to how we split everything into eigenspaces, the $n$-dimensional space gets split into

$k$ spaces like that, and if we plug A into $f(t)$, then $f(A) = $….. 0. (It’s just the characteristic polynomial with the exponents (multiplicities) stripped, and since those eigenspaces fill the whole space and each factor kills its own eigenspace, I think you can buy that it hits 0 without much trouble.)

In other words, if a polynomial evaluated at A gives the zero matrix, then the way $f(t)$ factors is what determines the ‘decomposition’ of the $n$-dimensional space.

Remember earlier when we defined that set called !? It was the set of all~ the polynomials that hit 0 when you plug A in for the variable.

If , then the decomposition of the matrix A is determined by how $f(t)$ factors!!

Reading this, a question probably popped up in your head:

“Hey -_- couldn’t there be a factor that doesn’t break down into first-degree pieces?” (Like, what are you gonna do if you’ve got something like ?)

The First Decomposition Theorem

OK, that’s where we have to start talking about the “First Decomposition Theorem.” Let me spoil the First Decomposition Theorem real quick and then prove it.

And if ($\gcd(g(t), h(t))$ being 1 means the two polynomials $g(t)$ and $h(t)$ have no common factor except constants like 1. In other words, the two are ‘coprime’: no common factor!!), then the $n$-dimensional space decomposes into kernels like

— and that’s the ‘First Decomposition Theorem.’

When you have a matrix like this, picking a suitable basis

turns it into this. That’s the claim.

It splits like this…. spoiler over. Now let me prove it.

Proof of the First Decomposition Theorem

$f(A) = 0$. $g(t)$ and $h(t)$ are coprime. If $g(t)$ and $h(t)$ are coprime, then there exist $x(t), y(t)$ satisfying

!! (Take a peek at the earlier post. It’s the same statement as: if integers $a, b$ are coprime then there exist $x, y$ with $ax + by = 1$.)

Take any vector X in the $n$-dimensional space, multiply it by 1, and write that ‘1’ all dramatically:

Now let’s plug A in for $t$.

Written this way, since $f(A) = g(A)h(A) = 0$, the red vector out front is a vector that becomes 0 when you multiply it by $h(A)$, and the blue vector behind is a vector that becomes 0 when you multiply it by $g(A)$!!!! So the front one dies under $h(A)$, meaning it lives in $\ker(h(A))$, and the back one dies under $g(A)$, meaning it lives in $\ker(g(A))$. Therefore,

And the reason we deliberately wrote direct sum is to stress that it’s determined uniquely. Since they’re coprime: for a vector sitting in both kernels, the red part and the blue part both come out 0, and they add up to the vector itself, so it can only be 0. That’s what pins the split down uniquely…..

An actual example

Let’s run an actual example to see whether this First Decomposition Theorem still works for matrices that can’t be diagonalized.

For this matrix A, let’s find an $f(t)$ with $f(A) = 0$. We can just use the characteristic polynomial (easiest path), so let’s go that route.

(Over the real numbers) you can’t factor it any further. OK so now let’s follow the First Decomposition Theorem.

This formula is supposed to hold, so let’s check it. Since

we get

like this. To think about each kernel, let’s find the vectors that become 0 when you multiply them on the right.

They’re linearly independent, see!~ Confirmed, so now let’s span.

Just like in diagonalization, we gather those vectors into R, and when we act on A like this, ,

we can confirm it does in fact end up like that.

Apparently that’s the power of blocking: like above, it can cut a quest in 4-dimensional space down to two 2-dimensional ones!!

---

Heads-up. The figures here are the original notes, slips and all. A few things to watch for:

• In the first characteristic-polynomial picture, $\det(\lambda I - A)$ should read $\det(tI - A)$, and in the general case $\phi_A(t) = (t - \lambda_1)\cdots(t - \lambda_n)$, not $(\lambda_1 - t)\cdots(\lambda_n - t)$ (they differ by $(-1)^n$).
• In the $4 \times 4$ example, the entry in row 4, column 2 of $A$ should be $-4$, not $4$; with $-4$, $\phi_A(t) = t^4 + 5t^2 + 4 = (t^2 + 1)(t^2 + 4)$ and $A^2$ come out exactly as shown.
• For $A^2 + 4I$, the first vector should be $(1, 0, 0, 1)$, not $(1, 0, 0, 0)$: $(A^2 + 4I)(1, 0, 0, 0) = (3, 3, 3, 3)$, while $\ker(A^2 + 4I) = \mathrm{span}\{(1, 0, 0, 1), (0, 1, 1, 1)\}$. The same vector appears in the $\mathbb{R}^4 = \mathrm{span} \oplus \mathrm{span}$ picture.
• In the final $A' = R^{-1}AR$, the lower block should be $\begin{pmatrix} 1 & 5 \\ -1 & -1 \end{pmatrix}$ (with $R$ built from $(1, 0, 1, 0)$, $(0, 1, 0, 1)$, $(1, 0, 0, 1)$, $(0, 1, 1, 1)$); on $\ker(A^2 + 4I)$ the matrix has to square to $-4I$. The upper block $\begin{pmatrix} 1 & -1 \\ 2 & -1 \end{pmatrix}$ is right.