Day 13: Rate Limiting

System Design with Data Structures & Algorithms ~9 min read 60 min hands-on

Welcome back, future system architects! Today, we're diving into a mechanism that often operates silently in the background, yet is absolutely critical for the stability, fairness, and even profitability of any high-scale system: Rate Limiting.

After our deep dive into the Auth Layer, you might think, "My users are authenticated, what else could go wrong?" Ah, my friend, even legitimate users can overwhelm a system, intentionally or not. That's where rate limiting steps in, acting as a bouncer at the club entrance, ensuring everyone gets a fair chance, and nobody ruins the party for others.

The "Why": Beyond Just DDoS Protection

When you first hear "rate limiting," your mind might jump to "DDoS protection." And while it absolutely helps with that, its true power in high-scale systems (think 100M RPS) goes much deeper.

  1. System Stability & Resource Control: This is the primary driver. Uncontrolled requests can exhaust database connections, CPU, memory, and network bandwidth, leading to cascading failures. Rate limiting ensures your backend services don't buckle under sudden spikes.

  2. Fair Usage & Quality of Service (QoS): Imagine a popular API. Without limits, one "greedy" client could hog resources, degrading performance for everyone else. Rate limiting ensures fair access, especially for multi-tenant systems or public APIs.

  3. Cost Management: Every request processed costs money – compute, storage, bandwidth. Rate limiting helps prevent excessive, unintended, or malicious usage that inflates infrastructure bills.

  4. API Monetization & Tiering: This is where it gets interesting for product managers and business strategists. Rate limits are often the backbone of API pricing tiers (e.g., 1,000 requests/month free, 100,000 for premium). It turns a technical control into a business lever.

  5. Preventing Abuse (Beyond DDoS): Scrapers, brute-force login attempts, spamming, or even just poorly written client-side loops can hammer your services. Rate limiting acts as a first line of defense.

The Non-Obvious Insight: While rate limiting protects your backend, it also costs you. Each rate limit check typically involves a read and potentially a write to a distributed key-value store (like Redis). At 100 million requests per second, this means 100 million additional operations on your caching layer before your actual business logic even runs. This "cost of protection" necessitates multi-tier rate limiting (e.g., edge-based limits on API Gateways/CDNs, then application-level limits for more granular control).

Where Does It Fit? The Overall System Architecture

Our rate limiter sits strategically in the request path, usually after authentication but before the core business logic.

Client -> Load Balancer -> Auth Layer -> Rate Limiter -> Application Service -> Database/Other Services

Why after Auth? Because you often want different rate limits for authenticated users (e.g., higher limits for premium users) versus unauthenticated guests. If it were before Auth, you couldn't differentiate.

Core Concepts: Demystifying Rate Limiting Algorithms

There are several common algorithms, each with its strengths and weaknesses. For a system handling 100M RPS, the choice often comes down to accuracy, memory usage, and distributed implementation complexity.

  1. Fixed Window Counter: Simple: count requests in a fixed time window (e.g., 100 requests per minute).

  • Pitfall: The "double-spending" problem. If a user makes 90 requests at 0:59 and 90 requests at 1:01, they effectively made 180 requests within a 2-minute period, but 90 in each 1-minute window, potentially exceeding the true rate. This can lead to bursts passing through. Avoid this for critical production systems.

  1. Sliding Window Log: Stores a timestamp for every request. To check, it counts requests within the last N seconds.

Keep going with the full lesson

The rest of this lesson — plus its implementation guide, runnable scripts and source code — is included with the course or an All Access plan. Create a free account to start the free lessons in every course.

Already a member? Sign in