The Batch Processor complements the real-time pipeline. Where the stream side optimises for freshness, the batch side optimises for completeness: it reprocesses full days of telemetry into curated, columnar datasets that power analytics and long-term archival.
Why batch as well as stream
Streaming answers “what is happening now?”; batch answers “what really happened?” — with late data folded in, corrections applied, and results written in an analytics-friendly format.
Could not build diagram for model "example-iot".
Triggers daily jobs and tracks their lifecycle.
Submits and supervises Spark applications.
Writes curated Parquet to object storage.
Operators and scheduled triggers submit, inspect and cancel jobs over the Batch Jobs API. The full contract is embedded on the Overview tab.