← All concepts
Scale & distribution
Horizontal scaling
Handle more load by adding more machines rather than bigger ones.
Stateless services can be copied freely behind a load balancer: any instance can serve any request because state lives in shared databases and caches. Autoscaling adds or removes instances as traffic changes.
See it in action
Step through the scenarios on the diagram.
Laying out diagram…
1/6Traffic doubles at lunchtime.
- Users
- — Traffic grows through the day
- Load balancer
- — Spreads requests across healthy instances
- Instance 1
- — Stateless
- Instance 2
- — Stateless
- Instance 3 (new)
- — Added by the autoscaler
- Session cache
- — Shared state lives outside the instances
- Metrics & autoscaler
- — Watches CPU and latency
How it works
- Keep services stateless; store sessions in a cache or tokens.
- Put a load balancer in front and autoscale on CPU, latency or queue depth.
- Stateful tiers (databases) scale out with sharding and replication instead.
Trade-offs
- More instances means more network calls and more moving parts to observe.
- Stateful components (connections, databases) don't scale this simply.