| ID | ingestion.cdc-sync |
|---|---|
| Description | Replicate operational DB changes to the lake in near real time. |
| Key | cdc-sync |
| Type | Sequence Workflow Diagram |
| Tags | ingestion storage |
Replicate operational DB changes to the lake in near real time.
End-to-end flow
This workflow is documented in specs under Workflows and embedded here as the canonical sequence diagram for the model.
| Number | Source | Destination | Description |
|---|---|---|---|
| 1 | Production DB | CDC reader | WAL/binlog connector to dest |
| 2 | CDC reader | Position tracker | Persist entities.LOG_POSITION |
| 3 | CDC reader | Kafka publisher | Emit services.CdcPublisher.PublishChange |
| 4 | Kafka publisher | Event bus | Publish channels.cdc.public.publish |
| 5 | Event bus | Stream router | Consume:channels.cdc.public.subscribe via dest |
| 6 | Stream router | Data lake | Write tables.bronze_cdc_orders schema subjects.cdcPublic.schema |