Phase 1: The Connection — a 10,000-socket WebSocket gateway in plain Java 21
A chat app's hardest moment isn't a busy one. It's the long, quiet stretch where ten thousand sockets are open, almost none are talking, and some are already dead without anyone knowing. This phase builds the gateway that handles that well: it holds the sockets cheaply, notices the dead ones, and lets a dropped phone pick up exactly where it left off.
What you build
A WebSocket server on raw
java.nio, with the HTTP upgrade and RFC 6455 framing written by hand.A Discord-style opcode protocol with heartbeats, identify, resume and precise close codes.
Two epoll reactors, a fixed worker pool with an ordered per-connection mailbox, and a timing-wheel reaper.
Sessions with a per-session replay ring buffer, so reconnects replay only what was missed.
A built-in load generator (10,000 real clients on virtual threads) and a live dashboard.
Key concepts
The Handshake and The Frame. The server turns an HTTP request into a socket, then parses frames byte by byte. Size limits come from the header before any allocation, and every malformed input gets its own close code (1002, 1003, 1007, 1009).
Protocol Design and Keep-Alive. One JSON envelope, {"op","d","s","t"}. Only dispatches carry a sequence number. Clients heartbeat on the interval from HELLO, with the first beat jittered so a reconnect wave doesn't beat in lockstep.
Zombie Killing. A hashed timing wheel holds every connection's deadline. The reaper looks at one slot per 250 ms tick instead of scanning everyone. In the demo, 330 silent clients were closed with 4009 about 7.3 s after going quiet, against a 7 s timeout.
Concurrency and Netpoll Optimization. A thread per socket cost 678 extra OS threads for 1,000 clients. Virtual threads and the NIO reactor each needed 2. The gateway uses one shared read buffer per reactor, so idle sockets hold no read memory.
Session State, The Resume Capability and The Replay Buffer. Sessions live in ConcurrentHashMap and outlive their sockets for 30 s. Each dispatch is numbered and buffered before it is sent. All 100 test clients that dropped mid-conversation resumed and got exactly their 10 missed messages each.
Serialization and Memory Profiling. The same message measured as JSON (237 bytes), ETF (204) and Protobuf wire format (93). A phantom-reference leak detector proves a forgotten listener pins 1,000 closed connections, and that the fix brings it to zero.
Architecture in one paragraph
An acceptor thread hands new sockets round-robin to two selector threads. Reactors handle the upgrade, decode frames and write queued bytes, and never parse JSON. Text messages go into a bounded, ordered mailbox per connection, which a fixed worker pool drains. Workers run the protocol, update the session store and assign sequence numbers. Each event goes into the session's replay ring before it is sent. A separate reaper thread advances the timing wheel, and a JDK HttpServer serves the dashboard and metrics.
What the dashboard shows
Header tiles for open sockets, live sessions, frames per second and heap. One panel per capability: handshake rejections by status, frames by opcode, live timing-wheel occupancy, the three-way concurrency table, worker queue depth, active vs detached sessions, resume and replay counts, serialization bytes and nanoseconds, a live load chart, and leak-detector results with a heap-dump button. During the load test, open sockets climb to 10,000 and fall back to zero.
Key takeaway
Treat the socket as disposable and the session as the asset. Number and buffer every event before sending it, and a dropped connection becomes routine instead of an incident.