← All components
Messaging
Message queue
Buffers tasks for workers; each message is processed once and removed.
When to use
- Background jobs, notifications, transcode requests
Pitfalls
- Add dead-letter queues and idempotent consumers
Options by provider
The first option is what diagrams show for that provider.
Amazon Web Services
- Amazon SQSManaged
- Amazon SNS — fan-outManaged
Google Cloud
- Pub/SubManaged
- Cloud TasksManaged
Microsoft Azure
- Azure Service BusManaged
- Azure Queue StorageManaged
Open source / self-hosted
- RabbitMQSelf-hosted
- NATSSelf-hosted
Related concepts
Used in
- Web crawler
- URL frontier — Per-host queues with priorities and next-allowed times
- WebSockets at scale
- Per-pod channels — Low-latency pub/sub to each gateway pod
- BookMyShow
- Virtual waiting room — Admits users to booking at a safe rate during spikes