| ID | product.processing.stream_processor.kafka_consumer |
|---|---|
| Description | Consumes telemetry topics. |
| Key | kafka_consumer |
| Type | component |
| Team | Stream Processing |
| Owner | stream-processing@iotgateway.example |
| Status | active |
| Technologies | Java, Kafka |
| Attributes | partitions: 128, throughput: 500k events/s |
| Tags |
Table of contents
Guide
Kafka Consumer
The Kafka Consumer is the entry point of the streaming pipeline. It reads the validated telemetry topics published by ingestion and hands records to the Window Aggregator with exactly-once semantics.
Offsets and checkpoints
Consumer offsets are committed as part of Flink checkpoints, not independently. On recovery the pipeline rewinds to the last checkpoint so no window is double-counted and none is lost.