Day 2: Add a cache and watch latency drop

In lesson 1 ShopStream topped out at about 300 product pages per second, because every page recomputed its rating from 200,000 reviews. Most of those computations produced the same answer as the one a millisecond earlier. That is the textbook case for a cache. In this lesson you add one with Redis, measure a 19× jump in throughput, and then break it the way caches really break in production: a popular key expires and fifty requests stampede the database at once.

The problem

Product pages are read hundreds of times for every time they change. Paying 6 ms of CPU for every read is waste. But "add a cache" is where most interview answers stop, and it's where the interesting questions start: what gets cached, for how long, what happens when the entry expires, and what happens when Redis is down?

What you build

start.sh adds three files and changes one:

• resp.py — a 60-line Redis client. Redis speaks a simple text protocol called RESP: an array of bulk strings in, a typed reply out. Writing the client yourself takes the magic out of redis-py; it does the same thing with more features. One connection per thread.

• cache.py — the cache-aside logic for product pages, with two modes.

• herd.py — the thundering-herd experiment.

• app.py — GET /products/<id> now calls cache.get_product(), and a new GET /stats reports hits, misses and database reads.

Cache-aside means the application owns the cache: read Redis first; on a miss, read the database and write the result to Redis with a time-to-live (TTL). The database never knows the cache exists.

The TTL is 30 seconds. That's a business decision, not a technical one: the rating and the displayed stock can be up to 30 seconds old. That's fine, because checkout doesn't trust the page — it runs the conditional UPDATE … WHERE stock >= qty from lesson 1 against the database. A stale page can't cause an oversell.

Measure the win

Same load test as lesson 1, now with the cache:

/stats after the run: 134,317 hits, 200 misses, 200 database reads — a 99.85% hit rate. The 200 misses are exactly the 200 hot products, each loaded once. The new ceiling is Python's HTTP server itself, not the database.