Multilinear Forms, Alternating Forms, and the Epsilon Symbol [Linear Algebra I Studied #12]

OK so up to this point we’ve been waving our hands and going “well, if the matrix is invertible…” and rolling the discussion forward from there. Now we’re actually going to figure out how to tell whether a matrix is invertible or not. You already saw it coming!!! The determinant!!!

To get there we first need to talk about something called a Multi-linear Form.

Heads up right out of the gate: this is not a linear function. lol I know, I know — “linear” is right there in the name, but nope, not a linear function. T_T Don’t stress about it, you’ll get the feel for it as we go!!!

Multi-linear Forms

OK. First let’s write an $n \times n$ matrix $A$ in terms of its $n$ column vectors.

A is written as the row of its n column vectors, each drawn in red inside its own brackets

We’re going to think of the matrix $A$ as an ordered tuple of $n$ vectors!!! Same vibe as writing out the coordinates of a vector in a specific order. And we’re going to define a function Ml. What does it actually do?

It eats those $n$ $n$-dimensional vectors as an ordered tuple and spits out ‘some number’.

Now now now now now — that’s not the part we need to take away from “Multi-linear Form”. The part we actually carry forward is the characteristic property of Ml. Ml is “a linear function with respect to any one column.” That is — $(X_1, \dots, X_n)$ — when we feed this ordered tuple into Ml and get a number out!!!!

What does “linear with respect to any one column” mean?

Let me show it with the first column, holding all the other columns fixed.

I showed it for the first column, but the same property works for the second, third, …, $n$th column — all of them, and a function with this property is called a Multi-linear Form.

So — see why Ml isn’t actually a linear function?!?!?!?

Here’s why it’s not a linear function (as soon as $n \ge 2$ — unless Ml is just the zero function, of course):

Scaling every column by c: Ml(cX) = c^n Ml(X), not c Ml(X)

Anyway, not a linear function, but it goes by the name Multi-linear Form. Sure, why not. Apparently there’s quite a variety of Multi-linear Forms out there. Same way there are loads of linear maps $L$ sending $V$ to $W$!!!!

Alternating Forms

Out of all the Multi-linear Forms, the one we actually care about here is the Alternating Form. Yeah, that name is doing a lot. lol

Anyway — since the Alternating Form is itself a Multi-linear Form, it inherits all the properties from above, and it tacks on one more property on top: “if you swap any two of the vectors, the Al value becomes (-) of what it was before the swap.”

That probably sounds abstract, so here’s an example with a matrix.

Al of a 3 by 3 matrix equals minus Al of the same matrix with its first two columns swapped

Now — there are also cases where swapping two columns leaves the matrix totally unchanged, in which case Al for that matrix shouldn’t change either. It’d be exactly a case like this.

The same equation with a red bracket under the left side and a blue bracket under the right side, each labelled ‘some number’

Right?? For both sides of that equation to come out as the same number — what on earth could ’that number’ up there possibly be?

Yep yep yep yep. It has to be 0. Both sides are some value, and the only way they can be equal even when one of them gets a (-) slapped on it is if both are zero (a number $x$ with $x = -x$ has $2x = 0$, and over the real numbers that forces $x = 0$).

So: when two columns of a matrix are the same, the Alternating Form value is 0!!!!!

(Many books define “alternating” by this “two equal columns give 0” property instead of the swap rule. Over the real numbers the two versions are equivalent.)

Oh, and just to be safe — since Al is itself a Multi-linear Form, all those properties from earlier still hold, right? Let me write them out before moving on.

Al(I) is all you need

OK so. Remember when we had a linear function $L$, and to fully pin down what $L$ does to anything, we just plugged the basis vectors into the map? Apparently the same kind of logic works for Al.

Al of a 2 by 2 matrix expanded column by column, with the terms that have two equal columns struck out as 0, ending in (ad - bc) times Al of the identity matrix

I used a $2 \times 2$ as the example, but the gist is: if you want to know what Al does to any matrix, you just need to figure out Al on the identity matrix, and you’ve cracked the whole thing!!!! Like —

L of the vector (x, y) equals x times L of (1, 0) plus y times L of (0, 1)

this!!!!!! Try it yourself for a $3 \times 3$ matrix!!!!!!

And for a general $n \times n$ you supposedly land at the same conclusion — you just need Al on the identity matrix (mathematical induction let’s go let’s go let’s go).

Al of a general n by n matrix written as a sum, over index tuples i_1 … i_n, of a_(i_1 1) … a_(i_n n) times Al of the columns e_(i_1), …, e_(i_n)

So we write it like this. Now — but since we’re going for full generality here, the numbers in $i_1 \sim i_n$ — don’t picture them as neatly going 1, 2, 3, 4, …, $n$ in order. Picture them as the numbers 1 through $n$ all jumbled up!!!!! ($n!$ possibilities; and if some number repeats, the matrix has two equal columns, so that term is 0 and just drops out)

Then take the $n \times n$ matrix at the very end,

The matrix whose columns are the standard basis vectors e_(i_1), e_(i_2), …, e_(i_n)

and try to massage it into the identity matrix I_n . To get to that nice clean identity-matrix shape,

The n by n identity matrix: ones on the diagonal, zeros elsewhere

we don’t yet know $e_{i_1} \sim e_{i_n}$ but we’re going to need some number of swaps to get there, right????

Whether it’s 0 swaps or $n$ swaps — if you make that clean shape with an odd number of swaps, a (-) gets attached (each swap flips the sign), and if you make it with an even number of swaps, a (+) gets attached, and I’ll punt on the exact count for now.

Anyway, the one value you get when you expand all that sigma stuff and add everything up — isn’t it just a constant multiple of

Al of the identity matrix ?

So here’s the focus for the next little while: if you just know Al(I_n) , you know every value of Al!!!

That’s the part we’re zooming in on. And so the Alternating Form with Al(I_n) equal to 1 is the one we call the Determinant!!!!

(Properly speaking — and this is a next-post thing — the Determinant is defined as the Alternating Form for which Al(I) = 1. Not the focus of this post.)

(Physics kids like me are going to be in absolute shock and horror — this is the rigorous definition of the determinant?? haha) (We’ll dig into it more in a bit, don’t panic.)

When the matrix isn’t invertible

Conclusion up front: if $A$ is non-invertible, then Al(A) = 0.

Let me explain why. Suppose the column vectors of $A$ are not all linearly independent. Then —

— this can be satisfied by some choice of $c_1$ through $c_n$ where they aren’t all zero. So at least one of $c_1, \dots, c_n$ is nonzero. If $c_1$ is nonzero (the same trick works for any other nonzero $c_i$), then —

The first column written as minus c_2/c_1 times the second column, and so on, up to minus c_n/c_1 times the last column

— we can also write the first column vector in this form…… and using that as our example,

The 3 by 3 matrix A with rows (6, 1, 4), (7, 1, 5), (12, 3, 6)

we write the matrix in this form, and then use the fact that the columns aren’t linearly independent to check the Al value.

Al(A) splits into 2 Al of a matrix with two equal columns plus Al of another matrix with two equal columns, both struck out as 0

So: if the column vectors are linearly dependent, Al(A) is zero, and therefore if Al(A) ≠ 0, the matrix is invertible. OK!!! So we can say: if Al(A) ≠ 0, then $A$’s columns are linearly independent!!!!! And if $A$’s columns are linearly dependent, then Al(A) = 0.

Now what about the converse???? That is, can we say that if Al(A) = 0 then $A$’s columns are linearly dependent??????

Yep, that’s the question. I dunno lol lol. For now let’s keep moving.

And let me mention one more property of Al. “The Al value of a matrix where you’ve added a constant multiple of one column to another column equals the Al value of the matrix before adding.” Isn’t that completely obvious????? (What if it’s not obvious to you…) hmmmm,,,

Al(X_1 + rX_3, X_2, X_3) expanded by linearity: the extra term has two equal columns, so it is 0 and the value is unchanged

Got it!?!? This plays a role later on….. we’ll meet it again when we hit the determinant.

From a concrete example to the epsilon symbol

OK that’s enough of that~~~ Let me actually compute Al for a $3 \times 3$ matrix concretely.

A general 3 by 3 matrix with entries a_11 to a_33

For this $A$, let me write out all the Al pieces we end up with.

The six terms of Al(A), each a product a_i1 a_j2 a_k3 times Al of a permutation matrix with its sign circled; notes on the right: after picking a_11, a_21 or a_31 there are 2 ways with no overlap, so 6 in all

I left off the plus + signs there. If you sum all those values up, you get Al($A$)!!! Now I want to construct

The 3 by 3 identity matrix

and bundle the whole thing up in one go~~

The three columns (1, 0, 0), (0, 1, 0), (0, 0, 1) side by side

I’ll think of the column vectors as labeled with the numbers 1, 2, 3 respectively. The number of swaps it takes to get an arbitrarily arranged 1, 2, 3 back into the order 1, 2, 3 — that’s exactly what decides whether the sign is (-) or (+), right?????? Something might be jogging your memory here: $\varepsilon_{ijk}$

We can express it with the epsilon symbol!?!? For folks who don’t know the epsilon symbol….. maybe you can pick it up from context here….. I’m just gonna roll with it lol lol lol lol. The epsilon symbol isn’t actually that hard a concept!!!!

How many swaps (a swap just exchanges two of the entries) does it take to get an arbitrarily arranged $i$ $j$ $k$ (some rearrangement of 1, 2, 3) into the order 1 2 3? If you can reach 123 in an even number of swaps, $\varepsilon_{ijk} = 1$ . If you can reach 123 in an odd number of swaps, $\varepsilon_{ijk} = -1$ .

For example, $\varepsilon_{123} = 1$ . 123 reaches 123 in 0 swaps. $\varepsilon_{132} = -1$ — 132 reaches 123 in 1 swap, so -1. $\varepsilon_{321} = -1$ — this one also reaches 123 in 1 swap.

Using this epsilon symbol, we can write all those expanded Al(A) terms as a single term.

Al(A) written with a sum over i, j, k of a_i1 a_j2 a_k3, times Al of the 3 by 3 identity matrix

We were talking about an arbitrary $3 \times 3$ matrix, but if we say the same thing for an arbitrary $n \times n$,

A general n by n matrix with entries a_11 … a_nn

Definition of the epsilon symbol: 1 if (1, 2, …, n) becomes i_1 … i_n by an even number of swaps, -1 if by an odd number

Al(A) equals the sum over i_1 … i_n of a_(i_1 1) … a_(i_n n) times epsilon_(i_1 … i_n) times Al(I_n)

Are you all squared away?????

Is the epsilon symbol well-defined?

The only thing nagging at us is whether the epsilon symbol is actually well-defined?!??!?!?!

So I think there are 3 things we need to check.

① No matter which $i_1 \sim i_n$ we start with, can we always get it into sorted order via some sequence of swaps?!?!? ↓ Yep, absolutely true.

When we start swapping $i_1 \sim i_n$ to get it into the order 1 through $n$,

First swap: send 1 to the very front.
Second swap: send 2 to the second slot.
Third swap: send 3 to the third slot.
…

Going like this, there’s no way we can’t end up at 1 through $n$.

② Once the permutation is fixed, is the number of swaps fixed too?!?!?!? ↓ Of course not!!! The count is not unique. You could do an efficient swap or an inefficient swap, right?!?!? But — there’s definitely one most-efficient way, no? The method I wrote in ① seems like it’d be the most efficient one.

③ For the epsilon symbol to actually work, the parity (odd-or-even-ness) of the number of swaps has to be guaranteed~~~? ↓ Apparently you get the answer by thinking of swap-parity as the number of inversions.

The number of inversions = “the number of pairs where a bigger number sits in front of a smaller one.” For example, to get the number of inversions of 7416253, let’s count the pairs where a bigger number sits in front of a smaller one:

(1,7), (1, 4), (2, 7), (2, 4), (2, 6), (3, 7), (3, 4), (3, 5), (3, 6), (4, 7), (5, 7), (5, 6), (6, 7)

Number of inversions = 13.

Once a particular permutation is given, the number of inversions is unique, right? If the inversion count is odd, then all of the many possible swap-counts are odd; if it’s even, all of them are even — that’s how you can think of it.

That is, the epsilon symbol behaves itself just fine, no malfunctioning.

Heads-up. The figures here are the original notes, slips and all. A few things to watch for:

In the picture that pulls $c$ out of every slot, the last line of the chain should read $c^n\,Ml(X_1, X_2, \dots, X_n)$, with every $c$ taken out; the picture leaves $c$ in front of $X_1$ and $X_2$. The conclusion $Ml(cX) = c^n Ml(X)$ is right.
In the picture that expands $Al(A)$, the sum needs the sign factor: $Al(A) = \left(\sum_{ijk} \epsilon_{ijk}\, a_{i1}a_{j2}a_{k3}\right) Al(I)$; the picture leaves out $\epsilon_{ijk}$.