← All components
Analytics & processing
Stream processor
Runs continuous computations (windows, joins, aggregations) over event streams.
When to use
- Surge pricing, fraud signals, trending topics, live dashboards
Pitfalls
- State and exactly-once semantics add operational complexity
Options by provider
The first option is what diagrams show for that provider.
Amazon Web Services
- Amazon Managed Service for Apache FlinkManaged
Google Cloud
- DataflowManaged
Microsoft Azure
- Azure Stream AnalyticsManaged
Open source / self-hosted
- Apache FlinkSelf-hosted
- Spark Structured StreamingSelf-hosted
Used in
- Twitter / X
- Trends — Counts hashtags in sliding windows per region
- Uber
- Surge computation — Supply vs demand per cell, every minute
- Search autocomplete
- Trending detector — Counts queries in short windows and boosts spikes
- Google Maps
- Traffic aggregation — Snaps points to roads and computes current speeds
- Stock brokerage
- Market data ingest — Normalises exchange feeds
- Live sports scores
- Match state processor — Applies events, updates stats