Day 25: Memory Limits

System Design with Data Structures & Algorithms ~10 min read 60 min hands-on

Welcome back, future system architects! Yesterday, we delved into unshare, a powerful tool for process isolation, giving our applications their own cozy namespaces. Today, we tackle another critical aspect of system robustness: Memory Limits.

You might think, "Memory limits? Just give my app enough RAM!" But in the world of ultra-high-scale systems, where 100 million requests per second is the norm and every byte counts, memory isn't just a resource; it's a strategic asset. Mismanaging it can lead to cascading failures that bring down entire services. This lesson isn't about saving a few MBs; it's about predictability, stability, and preventing the silent killer of distributed systems: the Out-Of-Memory (OOM) event.

Agenda: Mastering Memory Boundaries

  1. The Silent Killer: What happens when memory runs out?

  2. Core Concept: Linux Control Groups (cgroups) for Memory Management.

  3. Real-World Impact: Why memory limits are non-negotiable for production systems.

  4. Hands-on: Engineering a Memory Hog and Taming It with systemd-run.

  5. Practical Takeaways: Designing for Memory Constraints.

The Silent Killer: The OOM Killer

Imagine your application, humming along, suddenly needs more memory than the system can provide. What happens? Does it gracefully shut down? No. In Linux, a special mechanism called the OOM Killer springs into action. Its job is grim but essential: to find and terminate processes to free up memory, preventing the entire system from grinding to a halt.

While necessary, the OOM killer is indiscriminate. It might target your critical database, a vital caching layer, or even the shell you're using. The result is often an unexpected, abrupt termination of services, leading to downtime and data loss. This is why we don't just react to OOMs; we prevent them.

Core Concept: Linux Control Groups (cgroups) & Memory

The fundamental mechanism for enforcing resource limits in Linux, including memory, is Control Groups (cgroups). Think of cgroups as a way to organize processes into hierarchical groups and then apply resource constraints (CPU, memory, I/O, network) to those groups.

For memory, cgroups allow you to set:

  • memory.limit_in_bytes: The hard limit on RAM usage. If a process in this cgroup tries to allocate beyond this, the OOM killer is invoked.

  • memory.swappiness: Controls how aggressively the kernel swaps out memory pages for processes in this cgroup.

  • memory.memsw.limit_in_bytes: A combined limit for RAM + swap.

When you use technologies like Docker, Kubernetes, or even systemd to manage services, they are leveraging cgroups under the hood to provide resource isolation. This allows you to run multiple services on a single host, confident that one misbehaving application won't starve others of memory.

Real-World Impact: Beyond Just RAM

Why is this critical for systems handling 100M+ RPS?

Keep going with the full lesson

The rest of this lesson — plus its implementation guide, runnable scripts and source code — is included with the course or an All Access plan. Create a free account to start the free lessons in every course.

Already a member? Sign in