Day 5: Keep the database and events in step
Crash order-service between the database commit and the Kafka publish, count the orders that never get announced, and close the gap with a transactional outbox.
order-service now does two things for every order: it saves the order in its database, and it publishes OrderPlaced to Kafka. Two systems, two writes, no shared transaction. If the process dies between them, the database says the order exists and Kafka never hears about it: the customer is charged, the warehouse never ships. Today you make that crash happen on purpose, count the orders that fall through the gap, and close the gap with the transactional outbox.
What you build
A database for order-service: H2 in a file (
.run/orders-db), with tablesordersandoutboxand a sequence for order numbers.WRITE_DELAY=0makes every commit durable immediately β without it, a crash a few hundred milliseconds after COMMIT can still undo it, which our first run showed.Two ways to place an order (
POST /orders/batch {"mode": "dual"|"outbox", "count": N}):dual: commit the order, thenkafka.send().get().outbox: insert the order and an outbox row holding the event, in one transaction. Nothing is sent to Kafka here.OutboxRelay, a@Scheduledjob every 100 ms: read unpublished outbox rows in id order, publish each, then setpublished_at.CrashSwitch: with--app.crash.every-n=100the JVM halts (Runtime.halt, likekill -9) at every 100th crash point β right after COMMIT in dual mode, right after the send and before the mark in the relay.
The gap, in code
Reordering the writes does not help: publish first and a crash before COMMIT announces an order that does not exist. Wrapping both in @Transactional does not help either β Kafka is not part of the database transaction.
Read the full lesson free
This lesson is one of the first 3 in its course, free with an account β the full article, implementation guide and scripts. No card needed.
Already a member? Sign in