Day1: NumPy Playground

Lesson 1 60 min

The NumPy Playground

Python & NumPy for AI — From Raw Arrays to Vectorized Power

The `sklearn.fit()` Trap

Component Architecture

Here is how a beginner typically "does machine learning":

python

from sklearn.preprocessing import normalize
X_normalized = normalize(X)

One line. It works. And it costs you everything you need to know.

When you call normalize(X), scikit-learn quietly:

Checks your array's dtype and coerces it to float64
Computes the L2 norm along axis=1 using a C extension
Divides in-place using broadcasting across a reshaped norm vector
Handles the zero-norm edge case by clipping denominators

None of that is hidden because it's complicated — it's hidden because the
abstraction decided you don't need to see it. But every single one of those
steps is something that will break your custom model in a non-obvious way
the moment you step outside sklearn's guardrails.

This lesson makes those steps visible. You will implement them by hand,
watch them fail in predictable ways, and understand why the library made
each choice it did.

The Failure Mode

Flowchart

Here is the exact crash a beginner hits when trying to multiply two vectors:

python

import numpy as np

a = np.array([1, 2, 3])      # shape: (3,)
b = np.array([4, 5, 6])      # shape: (3,)

result = a @ b               # gives: 32  ← scalar, not a matrix!

No error. No warning. You get 32. If you expected a 3×3 outer product,
your entire downstream computation is silently wrong.

Or try this:

python

A = np.array([[1, 2], [3, 4]])   # shape (2,2), dtype int64
B = A / 3                         # you expect [0.333, 0.666, ...]
print(B)
# array([[0.33333333, 0.66666667],
#        [1.        , 1.33333333]])

That actually works — but only because Python 3 / forces float division.
Now try it with explicit dtype:

python

A = np.array([[1, 2], [3, 4]], dtype=np.int32)
B = A // 3
print(B)
# array([[0, 0],
#        [1, 1]], dtype=int32)

Floor division on an integer array. No error. Your gradients just became
integers. Your model trains silently on garbage.

These are not edge cases. They are the default behavior of NumPy, and
every neural network implementation lives or dies by getting them right.

The ScratchAI Architecture

State Machine

This lesson's module is the foundation layer of the entire ScratchAI
pipeline. Before you can write a forward pass, you need to reason fluently
about:

N-dimensional arrays: how NumPy lays them out in memory (row-major,
C-order by default), and why axis semantics matter for reductions
Shape algebra: every operation in a neural network is a shape
transformation — understanding (batch, features) @ (features, hidden)
is the entire forward pass, conceptually
Broadcasting: NumPy's mechanism for operating on arrays of different
shapes by virtually expanding dimensions — not copying data
dtype contracts: your model's numerical precision is a dtype decision
made at array creation time

The data flow in this lesson:

Code

Raw Input (Python lists / sliders)
    ↓  np.array(..., dtype=np.float64)
N-D Array [shape annotated]
    ↓  reshape / broadcast
Aligned Operands
    ↓  vectorized operation (dot, matmul, norm, outer)
Result Array [shape verified]
    ↓  dtype cast if needed
Output (displayed + explained)

No training loop yet. No loss function. Just the raw machinery that
everything else is built on.

Learning Objectives

✓ By the end of this lesson, students can:
✓ Create and inspect N-D arrays — understand shape, dtype, and memory layout (C-order)
✓ Apply broadcasting rules — predict which shape combinations work and why, without guessing
✓ Avoid dtype traps — recognize silent integer truncation and overflow before they corrupt a model
✓ Replace Python loops with vectorized ops — understand why NumPy is faster (C loop, SIMD, no Python object overhead)
✓ Write numerically stable code — use eps clipping, np.clip, and astype(float64) defensively

💬 Discuss this topic