| ID | product.storage |
|---|---|
| Description | Lakehouse — raw object storage layer plus a columnar warehouse for curated data. |
| Key | storage |
| Type | system |
| Team | Data Platform |
| Owner | storage@dataplatform.example |
| Status | active |
| Technologies | Iceberg, Parquet, S3 |
| Attributes | layers: bronze,silver,gold, pattern: medallion |
| Tags |
Storage is where data lives between ingress and consumption. It separates raw history (the bronze lake on object storage) from curated analytics tables (the gold warehouse) and holds the metadata — schemas, table catalog entries and snapshot lineage — that every other system references.
Responsibility in one sentence
Persist raw and curated datasets durably, manage partitions and snapshots, and expose authoritative schema and catalog metadata to pipelines and query engines.
Could not build diagram for model "example-data-processing".
The Data Lake accepts append-only writes from Ingestion. The Lake Writer partitions by time/key; the Compaction Job keeps query performance healthy.
CDC events land in the bronze layer as part of the sync flow:
Curated gold tables are materialized after the nightly batch completes:
1 incoming link · 3 outgoing links
| Source object | Label | Link type | Relation | Raw id | Required | Description | |
|---|---|---|---|---|---|---|---|
| lands data in | — | links | product.storage | — | — |
| Target object | Label | Link type | Relation | Raw id | Required | Description | |
|---|---|---|---|---|---|---|---|
| persists to | — | links | external_systems.object_storage | — | — | ||
| feeds | — | links | product.processing | — | — | ||
| serves curated data to | — | links | product.serving | — | — |