A reusable real-time layer: lightweight Go gateway pods hold millions of WebSocket connections behind an L4 load balancer. Backend services publish events to a durable bus; a router looks up which gateway pod each recipient is connected to and forwards the event over a per-pod pub/sub channel, which pushes it down the socket. Offline recipients' events are stored for when they reconnect.
Functional requirements
- Clients keep a persistent connection and receive events in real time
- Backend services can send to one user, a group (room) or everyone
- Presence: know who is online
- Deliver missed events after a reconnect
Non-functional requirements
- Millions of concurrent connections
- Delivery latency under ~100 ms
- Gateway pods can be added, removed or restarted without losing events
- Per-connection memory stays small
Extensions
- Server-Sent Events fallback for restrictive networks.
- Per-tenant rate limits on publishing.
- End-to-end delivery receipts surfaced to backends.