← All concepts
Reliability & correctness
Rate limiting
Cap how many requests a client can make in a time window to prevent abuse and protect capacity.
Every request is attributed to a client (API key, user, IP) and checked against a rule such as 100 requests per minute. Over the limit, the request is rejected with HTTP 429 instead of reaching the backend.
See it in action
Play with the idea, then follow it through a real architecture.
Try it: send requests
The bucket holds 5 tokens and refills one per second. Each request takes a token; an empty bucket means 429.
5 / 5
- Responses appear here.
Bursts up to the bucket size are allowed; after that the client is held to the refill rate.
Laying out diagram…
1/4A request arrives.
- API client
- — Sends requests with an API key
- API gateway
- — Asks the limiter before forwarding
- Rate limiter
- — Token bucket per client
- Counter store
- — Tokens left per client
- Backend
- — Protected service
How it works
- Token bucket: tokens refill at a steady rate; each request takes one; allows short bursts.
- Fixed window: count per minute — simple but allows 2× bursts at window edges.
- Sliding window: blends adjacent windows for smoother limits.
- Counters live in a shared in-memory store and are updated atomically.
Trade-offs
- Adds a check to every request; must be fast and fail open.
- Exact global limits across regions are costly; approximations are usual.