CDC sync from production
| ID | ingestion.cdc-sync |
|---|---|
| Description | Replicate operational DB changes to the lake in near real time. |
| Key | cdc-sync |
| Tags | ingestion storage |
Press enter or space to select a node. You can then use the arrow keys to move the node around. Press delete to remove it and escape to cancel.
Press enter or space to select an edge. You can then press delete to remove it or escape to cancel.
100%
Steps
Workflow steps
| Number | Source | Destination | Description |
|---|---|---|---|
| 1 | Production DB | CDC reader | WAL/binlog change |
| 2 | CDC reader | Position tracker | Persist offset |
| 3 | CDC reader | Kafka publisher | Emit event |
| 4 | Kafka publisher | Event bus | Publish |
| 5 | Event bus | Stream router | Consume |
| 6 | Stream router | Data lake | Write bronze layer |