The Metrics Pipeline turns counters, gauges and histograms emitted by every service into queryable time series. It is the backbone of dashboards and alerting — the first place on-call looks when something feels slow.
Pull, then remote-write
Prometheus scrapes targets on a fixed interval and remote-writes the samples to long-term storage. Local scraping keeps alert evaluation fast; remote write keeps history beyond a single Prometheus instance.
Prometheus pulls each target’s /metrics endpoint every 15s via service
discovery.
Recording and alerting rules run against freshly scraped data so alerts fire within seconds of a regression.
High-resolution samples are kept 15d locally; remote write hands older data to long-term storage for capacity planning.
Keep label cardinality bounded — per-device labels would explode the series count. Device-level granularity belongs in logs and traces, not metrics.