Day13:Layer Stack Builder

Build a Dynamic MLP from Scratch — Watch the Decision Boundary Move

The sklearn.fit() Trap

Here is how most beginners build a neural network classifier:

Four lines. It works. Accuracy prints. You move on.

But ask yourself: what happened inside fit()? Where are the weight
matrices? What shape are they? When a layer is added, what changes in
the gradient computation? How does the decision boundary of a
(64,)-wide single layer differ from a (32, 32) two-layer stack of
identical parameter count?

sklearn's abstraction doesn't just hide these answers — it actively
prevents you from asking the questions. The weights are buried in
clf.coefs_, the gradient computation is sealed inside a Cython
extension, and the training loop is a black box that either converges
or doesn't. When it doesn't, you have no instrumentation to know why.

This lesson rips that abstraction away. You will own every matrix,
every gradient, every numerical decision. The payoff isn't just
understanding — it's control.

The Failure Mode: Silent Symmetry Collapse

Before writing a single line of correct code, let's look at the most
common from-scratch failure. A student initializes all weights to zero:

The network trains. Loss decreases (slowly). But accuracy plateaus
around 50% on a problem that should reach 95%. The decision boundary
stays a straight line regardless of how many layers you add.

No error is thrown. NumPy never complains. The math is valid.
The result is silent garbage.