Implementing API Rate Limiting

Rate limiting in distributed systems requires consistent state management across instances, the right algorithmic approach, and clear communication with clients.

Overview

API rate limiting controls the number of requests a client can make to your API. In distributed systems, limits must be enforced consistently across all instances, which typically requires a centralized store like Redis or a database. The main algorithms are Fixed Window, Sliding Window, Token Bucket, and Leaky Bucket—each with different tradeoffs around fairness, burst handling, and implementation complexity. Key implementation details include client identification, atomic operations, status headers, proper error handling with 429 responses and Retry-After, and choosing appropriate limits. Rate limiting should be enforced both at the API Gateway or Load Balancer level for baseline protection and in application code for more nuanced, per-endpoint and per-user rules.

Core Components

Client Identification

Rate limiting needs a way to identify who is making requests. Common approaches include API keys, user IDs, IP addresses, or combinations of multiple factors. For authenticated APIs, user ID is usually most reliable since IP addresses can change and multiple clients may share a single IP.

Fixed Window

Fixed Window counts requests within discrete time intervals. In Redis, you store a counter per window and per client. The counter is created on the first request in that window and expires after the window closes. If the counter exceeds your limit, you reject the request.

The approach is simple to implement but can allow traffic spikes at window boundaries.

Sliding Window

Sliding Window is fairer than Fixed Window but more complex. One common variant is Sliding Window Log, where you maintain a timestamped list of all requests for each client. You remove timestamps outside the current window, then check if the remaining count exceeds your limit. If it does, you reject the request.

This prevents boundary spikes but requires more storage and computation.

Token Bucket

Token Bucket is widely used for its elegance and flexibility. A bucket has a maximum capacity and fills at a constant rate. Each request consumes a token; if the bucket is empty, the request is either rejected or queued. Redis with Lua scripts works well for atomic token operations.

This algorithm handles both sustained load and reasonable bursts.

Leaky Bucket

Leaky Bucket enforces a constant processing rate. Requests queue up and drain at a steady pace. It’s excellent for smoothing traffic but can introduce queuing delays and may discard requests if the queue fills.

Use this when you need predictable, uniform processing rates.

Redis for Distributed Rate Limiting

Redis is ideal for rate limiting: it’s fast, supports atomic operations, and offers built-in TTL. Lua scripts let you execute multiple commands atomically in a single round trip. Redis Cluster or Sentinel provide high availability. At very high request volumes, a single Redis instance can become a bottleneck, so consider local caching or proxy-level rate limiting in front of it.

Atomic Operations

Rate limiting must be atomic to avoid race conditions. If multiple concurrent requests each check “is there capacity?” without coordination, the limit can be exceeded. Lua scripts in Redis or database transactions ensure atomicity.

Status Headers

Clients need visibility into their rate limit status. Headers like X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset are widely recognized. The RateLimit spec (RFC 6585 and later RFCs) also defines standard patterns. These headers let clients adjust their request rate proactively.

Handling 429 Responses

When a client exceeds the limit, respond with HTTP 429 Too Many Requests. Include a Retry-After header to tell them when to try again. Format error details according to RFC 7807 (Problem Details) and avoid exposing internal details.

Per-Endpoint and Per-User Limits

Different endpoints have different costs. Write operations and expensive queries deserve stricter limits than simple reads. Likewise, different user tiers—free vs. paid, for example—can have different quotas.

Rate Limiting at the API Gateway

API Gateways like Kong, Nginx, or AWS API Gateway have built-in rate limiting. They protect your backend before requests arrive, making them effective for global limits and simple rules. Application code handles more complex, user-specific policies.

Practical Example

Here’s a Node.js service using Token Bucket rate limiting with Redis.

const redis = require('redis');
const client = redis.createClient();

const LIMIT = 100;
const WINDOW_SECONDS = 60;

async function isAllowed(clientId) {
  const key = `rate_limit:${clientId}`;
  const lua = `
    local key = KEYS[1]
    local limit = tonumber(ARGV[1])
    local window = tonumber(ARGV[2])
    local current = redis.call('GET', key)
    if current == false then
      current = 0
    end
    current = tonumber(current)
    if current >= limit then
      return 0
    end
    redis.call('INCR', key)
    if current == 0 then
      redis.call('EXPIRE', key, window)
    end
    return 1
  `;

  const result = await client.eval(lua, { keys: [key], arguments: [String(LIMIT), String(WINDOW_SECONDS)] });
  return result === 1;
}

Express middleware to apply the check:

async function rateLimit(req, res, next) {
  const clientId = req.user?.id || req.ip;
  const allowed = await isAllowed(clientId);
  if (!allowed) {
    res.set('Retry-After', String(WINDOW_SECONDS));
    return res.status(429).json({
      type: 'https://api.example.com/problems/rate-limit-exceeded',
      title: 'Rate Limit Exceeded',
      status: 429,
      detail: `You have exceeded the limit of ${LIMIT} requests per ${WINDOW_SECONDS} seconds.`
    });
  }
  next();
}

This approach is atomic, works across multiple server instances, and communicates limits clearly to clients.

FAQ: API Rate Limiting Implementation

1. How do you identify a client for rate limiting?

Clients are typically identified by API key, user ID, or IP address. For authenticated APIs, user ID is usually the most reliable approach.

2. Why is Redis a good fit for rate limiting?

Redis is fast, supports atomic operations and TTL, and is centrally accessible in distributed systems. Lua scripts enable complex atomic counting operations in a single call.

3. What is Token Bucket?

Token Bucket fills at a constant rate. Each request consumes one token. When the bucket empties, requests are rejected or queued. Redis with Lua scripts can implement this atomically.

4. What does atomic mean in rate limiting?

Atomic rate limiting ensures that checking and updating the limit happen as a single indivisible operation. This prevents race conditions and limit overages when requests arrive in parallel.

5. What is Sliding Window Log?

Sliding Window Log stores a timestamp for each request from a client. Timestamps outside the current window are discarded. If the remaining count exceeds your limit, the request is rejected.

6. What is Fixed Window?

Fixed Window counts requests in discrete time buckets. It’s simple to implement but allows traffic bursts at window boundaries.

7. What is Leaky Bucket?

Leaky Bucket queues requests and drains them at a constant rate. It smooths traffic effectively but can introduce queueing latency.

8. What is a race condition in rate limiting?

A race condition occurs when multiple concurrent requests check and update the limit simultaneously. Without atomic operations, the limit can be exceeded.

9. What headers should you set for rate limiting?

X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, and Retry-After inform clients about their quota and the next allowed request time.

10. What does 429 Too Many Requests mean?

429 is the HTTP status code indicating a client has exceeded its rate limit. Paired with Retry-After, it enables fair retry logic.

11. Should rate limiting happen at the gateway or in code?

Both are valuable. Gateways enforce global and simple limits before requests reach your backend. Application code handles complex, per-user and per-endpoint rules.

12. What are per-endpoint limits?

Per-endpoint limits apply to individual API endpoints. Write operations and expensive queries typically get stricter limits than simple reads.

13. What is a rate limiting bottleneck?

A bottleneck happens when the central rate limit store—like Redis—becomes overloaded. Solutions include local caching, proxy-level rate limiting, or optimized data structures.

14. What makes a good rate limiting strategy?

Combine reliable client identification, the right algorithm for your use case, atomic operations, informative headers, sensible per-endpoint and per-user quotas, clear error responses, and monitoring.

15. Why is Retry-After useful?

Retry-After tells clients when they can retry. This reduces wasted requests, lessens API load, and improves retry success rates.

Continue your API learning path

The next article in the API learning path covers Rate Limiting and Throttling for APIs — strategies and algorithms for rate limiting, and the differences between rate limiting and throttling.

References

Recommended books on API development

To deepen your understanding of rate limiting, distributed systems, and API architecture, we recommend the following books:

API Development

Books about API design, REST, GraphQL, OpenAPI and API architecture

Designing Data-Intensive Applications von Martin Kleppmann

Bei Amazon ansehen

Affiliate-Link: Bei einem Kauf erhalten wir möglicherweise eine Provision.

API Design Patterns von JJ Geewax