Day16:Data Cleaner

The sklearn.fit() Trap

Here is how most tutorials teach data preprocessing:

Four lines. It works. Ship it.

Except — what just happened? What does fit_transform actually compute?
What happens when an entire column is NaN? What happens when a column has
zero variance — all identical values? What happens when your numeric column
contains the string "N/A" that never got caught?

The answer: **sklearn silently produces NaN or inf in your feature matrix,
your model trains on garbage, and your loss curve looks fine because
the model learned to ignore the broken column.** You will not see an error.
You will see a subtly wrong model that fails on production data in ways
you cannot explain.

The abstraction hid the failure. This lesson removes the abstraction.

The Failure Mode: Silent NaN Propagation

Let's say you naively implement z-score normalization in a Python loop:

Two catastrophic failures: