Lesson 8-9: Supervised Learning: Regression

Introduction

Supervised learning curricula often teach regression and classification as separate exercises: gradient descent in one notebook, scikit-learn pipelines in another, a housing capstone here, a spam detector there. Each topic works in isolation. None of them behave like the systems engineers actually ship—train artifacts, expose predict endpoints, compare feature modalities, and run composed workflows behind one API.

week_8_9_aiml_integrated_project closes that gap. It is a self-contained capstone that unifies eight supervised-learning topics into one repository:

• An installable Python package (week89_python) holding domain logic for scratch regression, sklearn pipelines, multi-feature regression, housing prediction, logistic theory, binary text classification, multiclass strategies, and production spam detection

• A composition engine (supervised_ml_engine) that wires those topics into product workflows

• A FastAPI backend exposing two API surfaces: a lab path for per-topic exploration and a product path for composed regression and classification pipelines

• An optional React frontend—the Supervised ML Studio—for regression labs, classification demos, housing predict, spam operations, and pipeline runs

The goal is system-first learning. Gradient descent is not just a script—it is a stage inside regression_progression. Logistic theory precedes spam training in pipeline.run(). Topic sequence becomes control flow.

For engineers, the payoff is traceability. Housing predict is not a one-off function call—it is POST /api/v1/supervised/regression/housing/predict with a persisted scaler bundle, request IDs, and timing metadata. Spam modality comparison trains both TF-IDF text and tabular Spambase paths in one call, returning ROC-AUC side by side.

Core Components

The stack is layered. Each topic below is a distinct building block; together they form one coherent supervised-learning system.

Gradient descent linear regression

Implements simple linear regression from scratch using gradient descent on synthetic square-footage data. SimpleLinearRegression tracks weight, bias, and a downsampled loss_history suitable for charting.

The week89_apis module exposes JSON-safe api_d44_train, api_d44_predict, and api_d44_demo entry points. The supervised_ml_engine.regression.scratch facade wraps scratch training for engine callers. Lab routes accept n_samples, learning_rate, and iterations; responses return parameters, final MSE, and loss history.

Scikit-learn regression pipelines

Demonstrates salary prediction with scikit-learn: sample data creation, train/test split, model fit, and evaluation metrics. api_d45_pipeline_run and api_d45_predict serialize pipeline results for HTTP.

The supervised_ml_engine.regression.sklearn facade exposes run_salary_pipeline(). This topic bridges from manual gradient descent to library-based regression with preprocessing conventions.

Multiple linear regression

Generates realistic multi-feature synthetic datasets, trains a MultipleLinearRegressionModel, and returns train/test metrics plus feature-importance rankings.

api_d46_generate, api_d46_train, and api_d46_feature_importance power lab endpoints. The regression progression engine step chains scratch → sklearn → multi-feature before handing off to the housing capstone.

Housing price prediction

A production-style regression path: synthetic housing data generation, exploratory analysis, feature engineering (missing-value flags, derived room counts), train/validation/test splits, scaler persistence, and priced predictions from structured feature dicts.

HousingDataGenerator and HousingPricePredictor own the domain logic. supervised_ml_engine.regression.housing provides train_housing() and predict_housing()—auto-generating CSV data when missing. Artifacts land in backend/data/models/housing_model.pkl and backend/data/datasets/housing_data.csv.

Logistic regression theory

Covers sigmoid computation, log-loss calculation, threshold analysis, and linear-vs-logistic score mapping. SigmoidFunction, LogLossCalculator, DecisionBoundaryAnalyzer, and LogisticRegressionDemo implement the math.

supervised_ml_engine.classification.theory.sigmoid_bundle() composes sigmoid demo, linear-vs-logistic comparison, and threshold sweep into one product payload used by classification progression and full pipeline runs.

Binary text spam classification