Day5: Probability Sandbox

Probability Sandbox

AI Models From Scratch — Beginner Edition

The scipy.stats Trap

Here is how most tutorials teach probability:

Two lines. Correct output. Zero understanding.

What just happened? scipy.stats handed you a frozen distribution object
backed by decades of numerical routines. It computed the mean using a
closed-form formula it already knew. It never rolled a single die. It never
counted anything. And critically — it never showed you the gap between what
should happen in theory and what does happen in a finite sample.

That gap is the entire point of probability for machine learning.

Every training dataset is a finite sample from an unknown distribution. Every
loss function is an expectation over that sample. Every generalization bound
is a statement about how badly your finite-sample estimate can deviate from
the true distribution. If you do not viscerally understand the difference
between empirical and theoretical probability, you will misread loss curves,
misinterpret validation metrics, and make wrong calls on model capacity.

This lesson closes that gap — by building the probability engine from scratch.