Data Provenance & Telemetry Architect

General Diffusion, Inc.; San Francisco, Northern (CA, KY); Hybrid; USD 180,000 - 240,000; Full time

Job summary

General Diffusion, Inc. is seeking a Member of Technical Staff in Data Systems to build and evolve the measurement layer across heterogeneous machines.

You will ensure observations are comparable and traceable, enabling rigorous learning and analysis without owning model training or safety decisions. You’ll implement provenance, versioned data contracts, and reproducible evaluation workflows, collaborating with Runtime, Fleet, Research, and Safety teams.

Qualifications

• Experience building telemetry or data infrastructure for multiple producers and consumers.
• Strong judgment on backward-compatible data contracts and stable identifiers.
• Hands-on with lineage, versioning, manifests, and reproducible data workflows.
• Ability to validate data quality and diagnose system issues using SQL and Python.

Responsibilities

• Define and evolve versioned data contracts for workload shape, environment, and outcomes.
• Build ingestion paths for measurements across heterogeneous machines while preserving raw data and context.
• Attach provenance to datasets: instrumentation, configuration, and transformations.
• Produce reproducible snapshots and cohorts for training/evaluation with clear definitions.

Skills

Telemetry systems

Schema evolution

Data lineage

Reproducible pipelines

SQL

Python

Data quality validation

Operational diagnostics

Job description

General Diffusion, Inc. is seeking a Member of Technical Staff in Data Systems to build and evolve the measurement layer across heterogeneous machines.

You will ensure observations are comparable and traceable, enabling rigorous learning and analysis without owning model training or safety decisions. You’ll implement provenance, versioned data contracts, and reproducible evaluation workflows, collaborating with Runtime, Fleet, Research, and Safety teams.