Day 2: Keys, partitions and order

Every ShopStream order produces three events: OrderPlaced, OrderPaid, OrderShipped. Fulfillment must see them in that order — shipping an unpaid order is a refund waiting to happen. Kafka promises order, but only inside one partition. Today you choose the key that decides the partition, prove the right key keeps every order in order, and then use a plausible wrong key and count the damage.

What you build

• The orders topic now has 6 partitions, and fulfillment-service reads it with 3 listener threads (concurrency = 3), each owning two partitions.

• OrderEvent carries a type and a seq (1, 2, 3).

• POST /orders/simulate {"orders": N, "keyMode": "orderId" | "eventId"} publishes the three events of N orders back to back.

• OrderTracker checks, per order, that each event's seq is the last one plus one, and records which partitions an order's events used.

The key is the partition

Kafka's default partitioner hashes a non-null key with murmur2 and takes it modulo the partition count. Keys.partitionFor uses the same formula, so you can predict where an order will land:

Key by orderId and all three events of ORD-1001 hash to the same partition. One partition is an append-only log read by one thread, so they are consumed in the order they were written.

Key by eventId — a unique id per event, a common mistake when someone wants "even spreading" — and each event of an order lands on its own partition. Three threads now race through different partitions. Nothing in Kafka is broken; you simply asked it to put one order's history in three places.

Checking order on the consumer

An order counts as "shipped in order" only if all three events arrived 1, 2, 3.