Lesson 15-16: Reinforcement Learning & Other Topics

Introduction

Reinforcement learning and recommender systems tutorials usually ship as disconnected scripts: a grid-world Q-learning exercise in one folder, a MovieLens notebook in another. Each runs in isolation. None of them answer the question backend engineers actually face: how do policy training, hybrid movie recommendations, exploratory lab demos, and operator telemetry live together behind one API, one artifact layout, and one dashboard you could ship?

week_15_16_aiml_integrated_project closes that gap. It is a self-contained capstone that consolidates reinforcement learning and recommender systems into one repository:

• An installable Python package (week1516_python) holding domain logic for grid-world RL, collaborative filtering, content-based recommendation, adaptive hybrids, and model persistence

• A composition layer (core/) that picks canonical implementations for production paths—Q-learning trainers, SVD collaborative filters, genre+year content models, and adaptive blending

• Product workflows (product/) that run RL policy training and MovieLens hybrid recommend/train/serve paths end-to-end

• A FastAPI backend exposing two API surfaces: a lab path for per-topic exploration and a product path for composed ML workflows

• Adaptive Personalization Studio—a React dashboard behind nginx for live telemetry, training actions, registry inspection, and topic demos

The goal is system-first learning. Q-learning is not just a script—it is a train_q_agent call behind POST /api/v1/ml/rl/train with grid profile selection, policy JSON artifacts, and eval reward curves. Hybrid recommendation is not a notebook cell—it is dual .pkl persistence (collaborative + content) with adaptive alpha blending at serve time.

For engineers, the payoff is traceability. RL training and recommender fitting are timed HTTP calls with validation, model_name, request IDs, and last_train metadata recorded in runtime telemetry.

Core Components

The stack is layered. Each topic below is a distinct building block; together they form one coherent personalization system.

Grid-world environment and RL agents

ConfigurableGridWorld provides a unified grid environment with selectable profiles (day99 through day102 in the API) that vary grid size, obstacle layout, and reward shaping. QLearningAgent implements tabular Q-learning with epsilon decay; RandomAgent supplies a baseline for comparison. train_q_agent, compare_random_vs_q, and evaluate_agent in core/rl/trainer.py orchestrate training loops and greedy evaluation.

Q-learning lab demos

Learning modules expose RL introduction demos, random-vs-Q comparisons, tabular Q-table training with export, and grid-agent training with optional policy heatmap plots. The week1516_apis module wraps these as JSON-safe dicts—stripping non-serializable agent objects and embedding matplotlib plots as base64 PNG when needed.

Recommender theory and similarity engines

CollaborativeSimilarityEngine supports user-based and item-based collaborative filtering over synthetic rating matrices. StaticHybridRecommender blends collaborative and content scores for theory demonstrations. Learning routes surface CF similarity, hybrid static blending, and abbreviated pipeline previews.

Content-based filtering

ContentGenreRecommender fits genre one-hot features plus numeric release year, building a cosine-similarity matrix for item-to-item recommendations. A separate TF-IDF catalog demo runs over a small synthetic movie catalog for content-filter exploration without MovieLens dependencies.

Collaborative SVD and MovieLens loading

CollaborativeSVD factorizes the user-item rating matrix via truncated SVD (scipy.sparse.linalg.svds), producing a dense prediction matrix for rating estimation and top-N recommendations. MovieLensLoader reads MovieLens 100k (u.data, u.item, u.user), builds a sparse user-item matrix, and performs temporal train/test splits for holdout evaluation.

Adaptive hybrid recommender