Lesson 17-18: Advanced ML & Course Review

Introduction

Gradient-boosting tutorials usually end with a pickled model and a notebook cell. Production teams need more: repeatable experiments, tracked runs, persisted artifacts, and training code that does not fork into a dozen copies every time someone tunes a hyperparameter. week_17_18_aiml_integrated_project is a self-contained capstone that consolidates advanced ML topics into one runnable system:

• An installable Python package (week1718_python) holding synthetic fraud data, scratch gradient boosting, XGBoost/LightGBM trainers, bias–variance diagnostics, Optuna search, and inference benchmarks

• A FastAPI backend exposing two API surfaces: a lab path for topic exploration and a product path for experiment lifecycle management

• Experiment Workbench — a React dashboard with live run polling, learning demos, and artifact inspection

• Postgres-backed experiment metadata (async SQLAlchemy + Alembic), a dedicated Optuna RDB store, and joblib model artifacts on disk

The goal is system-first engineering. Scratch boosting is not just a script—it is a GradientBoostingClassifierScratch call behind a lab route with training-loss tails and feature importances. Hyperparameter search is not a blocking notebook cell—it is POST /runs/tune returning HTTP 202 while Optuna runs in a background thread, with the dashboard polling every two seconds until the run completes.

For engineers, the payoff is traceability: baseline training and Optuna campaigns produce Run rows with metrics_json, artifact paths, and study reference rows you can query without opening log files.

Core Components

The stack is layered. Each topic below is a distinct building block; together they form one coherent fraud-detection experimentation platform.

Synthetic fraud data generation

generate_transaction_data_v1 produces five interpretable transaction features (amount, time of day, location distance, frequency, merchant trust) with configurable fraud ratio—used by the scratch boosting lab demo. generate_fraud_data_v2 builds a higher-dimensional classification dataset via sklearn.datasets.make_classification with twenty features and a low fraud rate—used by library trainers, baseline runs, and Optuna tuning. Both live in week1718_python.core.data.synthetic.

Scratch gradient boosting classifier

GradientBoostingClassifierScratch implements binary classification boosting from first principles: residual targeting, shallow DecisionTreeRegressor weak learners, sigmoid outputs, and per-iteration log-loss tracking. Intended for pedagogy in the lab path, not as the default production model. Exposes feature_importances() for interpretability demos.

XGBoost and LightGBM trainers

train_xgboost_classifier and train_lightgbm_classifier wrap library classifiers with sensible defaults for imbalanced fraud data (scale_pos_weight / is_unbalance). predict_artifacts returns aligned y_pred and y_prob arrays. compute_scale_pos_weight balances class weights from training labels. These trainers power the library comparison lab endpoint, synchronous baseline runs, and the final fit after Optuna search.

Classification metrics

compute_binary_classification_metrics consolidates accuracy, precision, recall, F1, and optional ROC-AUC with zero_division=0—shared across lab demos, baseline runs, and tune jobs so metric shapes stay consistent in metrics_json.

Inference benchmarking

benchmark_predict_proba times predict_proba over a fixed batch across many iterations, returning mean latency, standard deviation, per-prediction milliseconds, and throughput per second. Used by the XGBoost vs LightGBM comparison endpoint to contrast model quality with serving speed.