| ID | product.processing.stream_processor |
|---|---|
| Description | Stateful stream pipelines (Flink / Kafka Streams). |
| Key | stream_processor |
| Type | microservice:service |
| Team | Stream Processing |
| Owner | stream-processing@iotgateway.example |
| Status | active |
| Technologies | Java, Apache Flink, ClickHouse |
| Attributes | latency: <1s, parallelism: 64, throughput: 500k events/s |
| Tags |
Table of contents
Guide
Stream Processor
The Stream Processor is the real-time heart of the platform. It reads the telemetry firehose from Kafka, maintains windowed state in Flink, and serves the resulting metrics and anomalies over a small read-only query API.
Why stateful streaming
Aggregates and anomaly scores depend on recent history per device. Keeping that state in Flink (checkpointed, partitioned by device) lets the pipeline answer “what is normal for this device right now?” without round-tripping to a datastore on every event.