Storage Engines & Indexing
Storage engines decide how data is laid out on disk and found again. These lessons build B-trees, LSM trees, write-ahead logs, compaction and secondary indexes from first principles.
Start free
Free lesson
Day 3: Persist Data to Disk with a Write-Ahead Log — and Corrupt It
Distributed Key Value Store : DurableKV
Free lesson
Day 2: Persist the Key-Value Store with a Write-Ahead Log — and Fake the Disk
Testing & TDD: Building Resilient Distributed Systems
Storage Engines & Indexing lessons by course
Hands-On Distributed Systems Engineering in Go
View course →- Day 30: Module Project: The “TurboShard” Index
- Day 54: Raft Log Compaction and Snapshots
- Day 61: The Write-Ahead Log (WAL) Internals
- Day 62: Log-Structured Merge (LSM) Trees
- Day 63: Memtables and SSTables
- Day 64: Bloom Filters for Faster SSTable Reads
- Day 65: Compaction Strategies: Size-Tiered vs. Leveled
- Day 71: Sparse Indexes for SSTables
The Relic Browser: A Production-Honest DSA Course
View course →- Day 4: Corrupt the Index File with Random Bit Flips — and Recover State using CRC32 Checksums
- Day 7: Merge the Search Index with the Graph Ranker — and Profile Stack Overflows under Concurrent Query Load
- Day 8: Crash the Graph Engine Mid-Update — and Restore Consistency with a Write-Ahead Journal
- Day 9: The Tension: If your application crashes or the power cuts out while you are updating your search index or graph structure, you will be left with a partially written, corrupted file. To guarantee durability, you must write changes to a append-only log before modifying the main database files.
- Day 10: The Hands-on Task: Implement a Write-Ahead Journal (WAJ) for your graph update operations. Before modifying the adjacency list on disk, write the intended changes to an append-only journal file and call fsync. Write a test that forcibly terminates your process mid-transaction, restarts it, parses the journal, and rolls back or completes the interrupted transaction to restore 100% consistency.
Distributed Key Value Store : DurableKV
View course →- Day 3: Persist Data to Disk with a Write-Ahead Log — and Corrupt ItFREE
- Day 8: Replicate Writes Asynchronously to Multiple Nodes with a WAL Stream — and Accept Divergence
- Day 12: Replicate the Write-Ahead Log to a Second Node — and Watch It Fall Behind
- Day 23: Implement Snapshotting and Compacting the Write-Ahead Log — and Limit Disk Usage and Recovery Time