Day3: Matrix Transformer

Lesson 3 60 min

The Matrix Transformer

Linear Algebra in Code — Dot Products, Transforms, and the Shape of Space

Course: AI Models From Scratch — Beginner Edition
Part: Course 1 of 3 | Difficulty: Beginner
Estimated Time: 90 minutes

The `numpy.linalg` Trap

Component Architecture

Here is how most tutorials teach matrix transformation:

python

import numpy as np
M = np.array([[0, -1], [1, 0]])          # 90° rotation
v = np.array([1, 0])
result = M @ v                            # [0, 1]
print(result)

That works. It is correct. And it teaches you almost nothing.

What it hides: why @ produces that result, what happens when you apply
M to 2,500 grid points simultaneously, what the number np.linalg.det(M)
is actually measuring, and — critically — what breaks when the matrix is
singular. The one-liner abstracts away the entire lesson. sklearn's
LinearRegression().fit() does the same thing to least-squares. Keras's
Dense(64) does it to weight initialization and matrix multiplication.

This lesson tears the abstraction open. You will implement the
transformation pipeline from scratch, apply it to a live point cloud, and
watch the grid of 2D space physically bend and stretch as you type matrix
entries into the app. By the end, M @ v will not be magic syntax — it
will be a computation you can expand, debug, and modify by hand.

The Failure Mode: Shape Mismatch in Matrix Multiply

Flowchart

Say you have 2,500 grid points you want to transform. The natural instinct:

python

M = np.array([[2, 1], [0, 1]])           # a shear matrix
points = np.random.randn(2500, 2)        # shape: (2500, 2)

result = M @ points                      # ← CRASH

Code

ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0,
with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 2500 is different from 2)

The rule of matrix multiplication: (m, k) @ (k, n) → (m, n). Your matrix
M is (2, 2). Your points array is (2500, 2). The inner dimensions are
2 and 2500 — they do not match. NumPy refuses.

The fix requires understanding why the shapes have to align. A 2×2
transformation matrix maps 2D vectors to 2D vectors: it expects its input
as a column vector of shape (2,), or as a batch of column vectors packed
into shape (2, N) — two rows, N columns. Your points array is the
transpose of that. So the correct call is:

python

result = (M @ points.T).T               # shape: (2500, 2) ✓

Or equivalently:

python

result = points @ M.T                   # same math, transposed convention

Both are correct. They express different mental models: the first says
"transform each column vector"; the second says "transform each row vector
using the transposed matrix." Understanding both is what separates someone
who can use NumPy from someone who can reason about it.

There is a second, more insidious failure. What if your matrix is singular?

python

M_singular = np.array([[1, 2], [2, 4]])  # det = 1*4 - 2*2 = 0
inv = np.linalg.inv(M_singular)

Code

numpy.linalg.LinAlgError: Singular matrix

A singular matrix has determinant zero. It collapses 2D space into a line
or a point — it destroys information, so it cannot be inverted. The app's
"Simulate Error" button loads exactly this matrix. Watch the entire grid
collapse onto a single line. That visual is the definition of determinant = 0.

The ScratchAI Architecture

State Machine

This lesson has no training loop and no loss function. Instead it has a
transformation pipeline: a pure function that maps a set of 2D points
through a matrix and returns the transformed coordinates. The computation
graph is:

Code

Grid Points (N, 2)
      │
      ▼
  Transpose → (2, N)
      │
      ▼
M @ P_T  →  (2, N)   ← this is the only "learning" operation in all of ML
      │
      ▼
  Transpose → (N, 2)
      │
      ▼
  Plot transformed coordinates

Supporting that core are four derived quantities, each computed directly
from M:

Quantity	Formula	What it measures
Determinant	`ad - bc`	Signed area scaling factor
Inverse	`(1/det) * [[d,-b],[-c,a]]`	The matrix that undoes M
Transpose	Rows and columns swapped	Reflects M across its main diagonal
SVD	`M = U Σ V^T`	Decomposes M into rotate→scale→rotate

Every function in model.py is pure: it takes arrays in and returns arrays
out, with no hidden state and no side effects. This is not a stylistic
choice — it is the foundation of every well-engineered ML system. The
Streamlit app calls these functions on each widget interaction and re-renders
the Plotly figure. That is the entire architecture.

Learning Objectives

✓ By the end of this lesson you can:
✓ Explain why M @ points crashes when points has shape (N, 2) and how points @ M.T fixes it
✓ Implement matrix multiply, determinant, inverse, and transpose using only NumPy — no linalg shortcuts
✓ Read a 2×2 matrix as a geometric instruction: where î and ĵ land tells you everything
✓ Use det(M) to predict whether a transformation preserves, scales, flips, or collapses space
✓ Guard against singular matrices with an epsilon threshold instead of exact zero comparison
✓ Transform a batch of N points in one BLAS call instead of a Python loop
✓ Decompose any matrix into rotate → scale → rotate using SVD

💬 Discuss this topic