The Export Writer is the final stage of a batch job. It takes the curated records produced by the Spark Runner and writes them as partitioned, columnar files into object storage for analytics and long-term archival.
Write format
Output is Parquet with snappy compression, partitioned by date. The columnar layout keeps downstream analytical scans cheap and the partitioning lets consumers prune to the days they need.
Results are written to a temporary prefix so a partially written partition is never visible to readers.
On success the staged files are atomically promoted to the final date partition, replacing any prior output for an idempotent rewrite.
The new partition is recorded so analytics consumers can discover it.