Lesson 2-3: Linear Algebra & Calculus Essentials

Introduction

Most engineers learn linear algebra and calculus as isolated exercises: dot products on paper, partial derivatives in notebooks, gradient descent as a one-off plot. That knowledge rarely connects to how production systems actually run—HTTP handlers, validation, training loops, and structured responses.

week_2_3_aiml_integrated_project closes that gap. It is a self-contained capstone that unifies vector math, matrix operations, derivatives, and optimization into a single ML Engine: an installable Python package composed of topic modules, exposed through FastAPI and an optional React lab UI. The goal is not to replace frameworks like PyTorch, but to show how fundamentals become a coherent system you can reason about, test, and extend.

For engineers, the payoff is traceability: cosine similarity is not just a formula—it is a function with dimension checks, pytest coverage, and an HTTP contract. Gradient descent is not just a plot—it is an epoch loop that returns weight, bias, and loss_history as JSON.

Core Components

The stack is layered. Each topic below is a distinct building block; together they form one coherent ML system.

Vector similarity and recommendations

Represents users and catalog items as vectors in a shared feature space. Dot product measures alignment (magnitude-sensitive). Cosine similarity normalizes by vector length, emphasizing direction over scale.

In the ML Engine, similarity.recommend() scores each catalog item against a user vector, sorts by score, and returns the top n results. Dimension mismatches raise explicit ValueError messages before scoring begins.

Key functions: cosine_similarity, dot_product_score, recommend.

Linear regression model

Applies the line y = w·x + b to feature vectors. The LinearModel dataclass wraps a simple predictor with weight and bias fields; predict() and mse_loss() expose inference and error measurement.

This component powers POST /api/v1/ml/predict—apply learned parameters to new features without retraining.

Gradient descent optimizer

Minimizes mean squared error by updating weight and bias each epoch. compute_gradients and gradient_descent_step expose single-step mechanics; train_linear runs the full loop and returns loss_history.

Training is the stateful operation in the product layer: the handler stays in-process until all epochs complete.

Neural network forward pass and backpropagation

A layer is a batched linear transform Y = XW + b, optionally followed by ReLU. nn.forward() returns both pre-activation (linear) and post-activation (output) tensors.

train_step_forward_backward() runs one forward pass on a small two-layer sigmoid network, computes gradients via the chain rule, and applies a single SGD update—mirroring how depth stacks differentiable operations.

Pipeline orchestration

pipeline.explain_system() returns structured metadata for GET /api/v1/ml/explain. run_demo_pipeline() chains recommend → train → predict → neural forward into one response for smoke testing.

Math lab modules (domain package)

The installable package week23_python holds standalone topic modules—linear algebra, vector similarity, matrix operations, matrix multiplication, calculus toolkit, derivatives, chain rule, gradient visualization, and math review. Each module is importable and testable on its own.