Change of Basis

OK so, let me set up one general situation.

Here’s our linear map: $L: V \to W$, where $V$ and $W$ are finite-dimensional vector spaces over the same field (I’ll write $\mathbb{R}$ for the scalars, but any field works the same way), with $\dim V = n$ and $\dim W = m$ (it’s right there in the picture). Each space gets an ordered basis: $\beta = \{v_1, \dots, v_n\}$ for $V$ and $\gamma = \{w_1, \dots, w_m\}$ for $W$.

Now I really want to actually understand what this thing is doing. From the very start I’ve been preaching the same gospel: if you want to understand a linear map, just track where the basis vectors go. Once you know that, you know everything~~~ about where any element ends up.

So here’s how I’ll write it:

Let $a_{ij}$ be the entries of the $m \times n$ matrix $A$, and let’s say this matrix $A$ represents the linear map $L$ in the bases $\beta$ and $\gamma$!! In formulas: $L(v_j) = \sum_{i=1}^{m} a_{ij} w_i$ for every $j$, so column $j$ of $A$ is the coordinate column of $L(v_j)$ in $\gamma$ (read the indices in the picture that way). Cool? Cool.

Now now now now — take some element $v$ in $V$. Expand it in the basis $\beta$:

And some element $w$ in $W$, written using the chosen basis $\gamma$ of $W$ (that’s $m$ terms, since $W$ is $m$-dimensional):

That’s how we’ll write them.

And — suppose we found another basis $\beta'$ for $V$ besides $\beta$. And let’s say we also found another basis $\gamma'$ for $W$ besides $\gamma$.

Then obviously we can write things this way too, now with the primed bases: $L(v_j') = \sum_{i=1}^{m} a'_{ij} w_i'$.

Obvious, right?!?! The matrix with entries we’ll call $A'$. And $A'$ also represents the linear map $L$!!!! (Same $L$, same $V$ and $W$. Only the bases changed, to $\beta'$ and $\gamma'$.)

So now — the relationship between $A$ and $A'$. (Let’s derive $A'$ from $A$!!!) Hold up, before that — let me give you a little more intuition.

Same linear map, different matrix

The same linear map can get a different matrix when you change the basis.

OK, but how different? In what way?

Here’s one example. Take a left-right reflection map $L$ (the mirror across the $y$-axis, $L(x, y) = (-x, y)$), and let’s see how the matrix shape changes depending on which basis we pick.

The point after the reflection becomes , where are, respectively, . So the matrix $A$ representing this $L$ looks like:

<When using the standard basis>

(See why the matrix has that shape now?! Take the column vector before the map, hit it with the matrix, and yeah — out comes the reflected point!!)

Now let me tweak the basis a bit and run it through the same reflection map.

Not the standard basis — a slanted, crooked one (the axes tilted by 45°, as you can see). Convince yourself that the matrix $A'$ representing the same linear map, with this basis used for the input and for the output, is

right?!?!

Same reflection map. Different basis. Different-looking matrix.

Coordinate transformation

OK, so — depending on how the basis changes, how does the matrix shape change? Hmm, well… let’s see if there’s a clean connection. This is “coordinate transformation” territory.

What does it even mean to express a point $P$ in a different basis?

Well, two different expressions → two coordinate tuples, which will usually differ (the zero vector, for one, gets all zeros in every basis). Remember what a coordinate is? When you write some element in terms of the vectors of an ordered basis, the ordered tuple of scalars you multiply each basis vector by — that’s the coordinate.

Let’s give those tuples a name. Say $f: V \to \mathbb{R}^n$ is the function that sends each element of $V$ to its coordinate tuple in $\beta$, and $f'$ is the same thing for $\beta'$ (both are bijective linear maps, isomorphisms, just like last time). In the picture the lower arrow should be labelled $f'$.

What this picture is saying: whether you write element $v$ in $V$ using the $v_i$’s, or using the $v_i'$’s —

— there’s a relationship between the two coordinate tuples!! Because the two bases are related to each other!! Each element of $\beta'$ is itself an element of $V$, after all. (Both bases are linearly independent and span $V$! Don’t forget~)

A quick note on index letters: I’ll write $v_j' = \sum_{i=1}^{n} r_{ij} v_i$ for each $j$, so that column $j$ of the matrix $R = (r_{ij})$ is the coordinate column $[v_j']_\beta$. The handwritten notes below are a bit loose with the two indices of $r_{ij}$; when in doubt, go by this formula.

Let me just check the first row.

Get it?! One element of $\beta'$ can be written as a linear combination of the elements of $\beta$ (with unique coefficients, because $\beta$ is a basis). And the ordered tuple of coefficients in that linear combination is — yep — a “coordinate”. Stack that coordinate as a column vector and you get $[v_j']_\beta = (r_{1j}, \dots, r_{nj})^T$:

— that’s all this is saying~~~ Anyway, big idea: every element of $\beta'$ is a linear combination of the elements of $\beta$.

So what we need to take away here is

— this matrix right here, $R = (r_{ij})$, is exactly the coordinate transformation matrix. It turns coordinates in $\beta'$ into coordinates in $\beta$, new into old: $[v]_\beta = R\,[v]_{\beta'}$. And how do you build it?

You just stack those column vectors we found above, side by side~~

In plain English: take the basis vectors of the new coordinate system (the one you want to switch to, the “target” system, here $\beta'$), express each of them as coordinates in terms of the existing, old basis $\beta$, and then stack them as column vectors, one after another, in order, neatly. Boom — there’s your matrix. (Mind the direction: this $R$ takes new coordinates to old ones. Going from old to new is $R^{-1}$, which exists, as we’ll see.)

Let me sanity-check this with a rotating coordinate system.

If you want to switch to the coordinates of a rotating frame, does stacking the basis vectors of the rotating frame as column vectors really give you the coordinate transformation matrix you already know?! Or is it total nonsense?!

Let’s check!!

The new basis is the $\beta'$ in here (the rotated one), and stacking those one by one should give the coordinate transformation matrix — yeah~~? Let me stack them.

Whoa?!

It really is the coordinate transformation matrix we already know, the rotation matrix, the one that turns coordinates in the rotated frame into coordinates in the standard one!!

How clean is that?!

Deriving A’ from A

OK now I’m switching to completely~~~ different notation and starting a new discussion. Throwing this picture down:

Inside $V$ space, matrix $R$ is doing a coordinate transformation between the $\beta$ basis and the $\beta'$ basis: $[v]_\beta = R\,[v]_{\beta'}$. Same deal in $W$ space, with the matrix $S$: $[w]_\gamma = S\,[w]_{\gamma'}$. That’s what the picture is telling you. ($R$ is $n \times n$ and $S$ is $m \times m$, so on the $W$ side the picture’s two $\mathbb{R}^n$ should be $\mathbb{R}^m$.) And we’ve got the function $L$ connecting $V$ and $W$, and the matrix representing it is either $A$ or $A'$.

Let’s set $L(v) = w$. If we want to express that $w$ using the $\gamma$ basis — well, following the picture above, we should write $v$ in the $\beta$ basis, and then push it through matrix $A$!! As a formula:

Look at the picture above and you’ll see why this formula works.

Now — about $A$. Let me show you how to write, formula-style, the statement “matrix $A$ represents linear map $L$, and it’s nailed down with bases $\beta$ and $\gamma$"~~~

Don’t forget: the map goes from the superscript basis to the subscript basis.

If we plug this notation into the formula above:

That’s how it gets written.

Now let’s do the same thing for $A'$. Same principle:

That’s it. So:

Now I’ll rewrite the left side as a coordinate transformation inside $W$ space, and the right side using a coordinate-transformed basis inside $V$ space. (Hard to put in words, so let me just throw up a picture. One fix: the left side of the second line needs its $S$, because $[L(v)]_\gamma = S\,[L(v)]_{\gamma'}$. So it reads $S\,[L(v)]_{\gamma'} = [L]_\gamma^\beta\,R\,[v]_{\beta'}$.)

Now move $S$ over to the right side as its inverse — like this!!!

Wait!!

What if matrix $S$ has no inverse?!

↓

Don’t worry, not even a little: $S$ is the matrix of an isomorphism, so of course it’s invertible!!!

It’s the identity map of $W$, written with $\gamma'$ going in and $\gamma$ coming out, and the inverse exists, guaranteed~~~ (Concretely: the columns of $S$ are the coordinate columns of $w_1', \dots, w_m'$, which are linearly independent because $\gamma'$ is a basis, and $m$ independent columns in an $m \times m$ matrix make it invertible. Same story for $R$.)

Anyway — since $S$ and $R$ are coordinate transformation matrices, we can think of them as identity maps (copy maps), apparently. To be precise, they are the matrices of the identity map $v \mapsto v$ (on $V$ for $R$, on $W$ for $S$), just written with two different bases: $R = [I]_\beta^{\beta'}$ and $S = [I]_\gamma^{\gamma'}$.

Careful with the direction here: $R$ takes us from prime coordinates to normal ones, and $S$ does exactly the same job in $W$!! (The first picture has the two indices of $R$ swapped; it should read $R = [I]_\beta^{\beta'}$, like $S = [I]_\gamma^{\gamma'}$.) It’s $S^{-1}$ that goes the other way, from the normal coordinate system to the prime one!! So the formula above is!!

Read it from right to left: first $R$ (prime to normal coordinates in $V$), then $A$ (from $V$ to $W$), then $S^{-1}$ (normal to prime coordinates in $W$).

This stuff we just did? It comes back later. Hard. Don’t!! Don’t ever!! Treat it carelessly.

So to make it land cleanly when it shows up again — let me organize one more thing I haven’t said yet, in the form it’ll get used later.

Remember this picture?

Focus right here:

Now —

Look at it like this:

Multiplying an invertible matrix on the left of $A$ — what was that doing?? What was the role of $S^{-1}$?

It was changing the basis of $W$ — the target space (the codomain). And multiplying an invertible matrix on the right of $A$ — what was that one doing? What was $R$’s job?! Yep — changing $V$’s basis. Right?!?!

This is a theorem that comes back later, so lock it in!! (Here $A$ is the $m \times n$ matrix of a linear map $L: V \to W$ between finite-dimensional spaces.)

• Multiplying an invertible $n \times n$ matrix on the right of $A$ → changing the basis of $V$.
• Multiplying an invertible $m \times m$ matrix on the left of $A$ → changing the basis of $W$.

(Same linear map all along; only the bases change. And every invertible matrix of the right size can play this role: its columns are linearly independent, so they are the coordinates of a new basis.)

And the thing you must absolutely, absolutely never forget!! How do you build a coordinate transformation matrix?!

Recipe for $R$: take the basis vectors of the new (“target”) basis, write each one in coordinates of the old basis, and stack them, neatly, in order, as column vectors. That’s the matrix that turns new coordinates into old ones.

Please — keep this one in your head!!

---

Heads-up. The figures here are the original notes, slips and all. A few things to watch for:

• In the two pictures that define the matrix entries, the sum runs over $i$: $L(v_j) = \sum_{i=1}^{m} a_{ij} w_i$, and $L(v_j') = \sum_{i=1}^{m} a'_{ij} w_i'$ for the primed bases (the second picture also drops the prime on $v_j'$).
• In the picture that expands $w$, the sum ends at $x_m' w_m$: $W$ is $m$-dimensional.
• In the picture with the two arrows to $\mathbb{R}^n$, the lower arrow should be labelled $f'$; the picture writes $f$ twice.
• In the second line of the picture that changes the basis of $V$, the left side should be $S[L(v)]_{\gamma'}$ (that is, $[L(v)]_{\gamma}$): the $S$ is missing.
• In the picture that writes the new basis vectors as a column, the column ends at $v_n'$ ($V$ has $n$ basis vectors); the picture writes $v_m'$.
• In the picture that expands $A'$, the first line should read $A' = [L]_{\gamma'}^{\beta'}$; the picture drops the prime on $\gamma$.