Day14:Weight Watcher

The sklearn.fit() Trap

Open any beginner ML tutorial and you will see something like this:

It works. It gives you a loss curve and a test score. But here is what it
hides from you: MLPRegressor internally calls _initialize_weights(),
which uses Glorot uniform initialization (Xavier) by default. That single
decision — buried three call-stack frames below .fit() — is what allows the
gradients to flow. Remove it, and the same network trains to nothing.

The library made a wise engineering choice on your behalf, invisibly. If you
never look behind .fit(), you will never know that initialization exists,
which means when your custom network fails to train you will spend hours
tuning the learning rate when the real problem was set on line 1.

This lesson makes that invisible decision visible.

The Failure Mode

Let's start a two-layer network with all weights set to zero and run it for
50 epochs. Here is what you get: