Lesson 10: Stateful Stream Processing & Interactive Queries

Hands On Kafka Course ~4 min read 60 min hands-on

StreamSocial Stateful Processing Platform

Kafka Mastery: Building StreamSocial — Event-Driven Systems

Build Scope

  • Event enrichment — join live user actions with profile materialized views.

  • Recommendation matching — table-table joins on preferences and content metadata.

  • Trending query surface — windowed RocksDB state exposed through REST.

  • Engagement scoring — changelog-backed state with decay and recovery.

  • Content scoring topology — custom processor chain with weighted ranking.

  • Operations console at http://localhost:5100 — live metrics and recovery controls.

Platform Position

StreamSocial already ingests events and runs stateless transforms elsewhere in the estate. This repository is the stateful compute plane: where raw activity becomes enriched events, matchable recommendations, queryable trends, and durable engagement scores. LinkedIn's feed mixer, X's trending rail, and Spotify's ranker all compile to the same primitives implemented here.

Core Concepts

Stream-table join avoids the database round-trip

A like event carries user_id and target_id only. A co-partitioned materialized view keyed by user_id holds demographics. The join runs in-process—the same pattern Instagram applies before feed ranking.

Table-table join reacts to either table changing

A preference update or a new content publish both recompute match scores. At moderate catalog sizes a scan-on-update model is acceptable; at scale you shard by category or co-partition.

Keep going with the full lesson

The rest of this lesson — plus its implementation guide, runnable scripts and source code — is included with the course or an All Access plan. Create a free account to start the free lessons in every course.

Already a member? Sign in