SpecsModelExploreRequirements
SpecsModelExploreRequirements
Data Engineer
Overview
Guide
Twin streams
Links and Communications
IDactors.data_engineer
DescriptionBuilds processing pipelines and exports for the data team.
Keydata_engineer
Typeactor
Statusactive
Tags
Table of contents
Guide
Data Engineer
Goals
Where they work
A pipeline they own
Links and Communications
Requirements

Attributes

accesspipelines
rolebuilder

Guide

Data Engineer

The Data Engineer turns the firehose of device telemetry into useful, queryable data. They build and own the stream and batch pipelines, and the exports that downstream analytics teams depend on.

What success looks like

Reliable, well-shaped data: clean topics, correct enrichment, and exports that land in object storage on schedule and without gaps.

Goals

  • Author stream-processing jobs that enrich and route telemetry.
  • Define windowed aggregations and alert thresholds.
  • Ship batch exports to the data lake.

Where they work

They build inside Processing and land results in Object Storage.

A pipeline they own

The alert pipeline is a canonical example of the stream work they build and maintain — from raw telemetry to a routed, deduplicated alert:

Links and Communications

1 outgoing link

Outgoing Links(1)

Target objectLabelLink typeRelationRaw idRequiredDescription
Stream Processing

example-iot.product.processing

system
builds pipelines in—linksproduct.processing——

Requirements