Day 28: Fit Visualizer

The Fit Visualizer — Overfitting, Underfitting, and the Bias-Variance Tradeoff

The sklearn.fit() Trap

Here is how a beginner solves polynomial regression in scikit-learn:

Four lines. It works. And it hides everything this lesson is designed to teach.

What is PolynomialFeatures actually doing? It is expanding your raw input x into a matrix of powers: [1, x, x², x³, x⁴, x⁵]. That matrix is called a Vandermonde matrix, and its shape — (n_samples, degree+1) — is the first thing you need to reason about when debugging a shape mismatch. sklearn builds it silently. You never see it.

What does LinearRegression().fit() actually solve? It minimizes the mean squared error by solving the normal equation: w = (XᵀX)⁻¹ Xᵀy. Except it does not actually invert XᵀX — it uses a numerically stable least-squares solver internally because matrix inversion at high polynomial degrees fails catastrophically. You never see that either.

When degree=12 and your model fits the training data perfectly but produces wild oscillations on unseen data, sklearn gives you no signal. It just returns a score. This lesson makes that failure mode visible and interactive.

The Failure Mode: What Breaks When You Do This Naively