Data Engineer: End-to-End Physics Data Pipelines

Causal; San Francisco (CA); On-site; USD 120,000 - 160,000; Full time

Job summary

Causal, a San Francisco startup, is building a Large Physics foundation Model to learn physics from sensory data. We seek data engineers to own datasets end-to-end—from source discovery to training-ready pipelines and access controls—ensuring data quality for multimodal physical data.

You will design scalable pipelines using Spark, Ray, and Beam, develop QA checks, and collaborate with researchers to verify that datasets translate into model performance.

Qualifications

• Experience building large-scale data pipelines, QA systems, or evaluation workflows.
• Ability to identify data quality issues and their impact on model performance.
• Experience reading sensor data specifications and metadata for ingestion.

Responsibilities

• Own dataset end-to-end from discovery, access, ingestion, to training-ready data.
• Build petabyte-scale data pipelines and orchestration, storage, and monitoring.
• Develop quality metrics to measure coverage, correctness, and consistency.
• Design and implement automated QA checks to monitor data quality over time.
• Collaborate with researchers to ensure datasets improve model performance.

Skills

Large-scale data pipelines

QA systems

Evaluation workflows

Tools

Apache Spark

Ray

Beam

Job description

Causal, a San Francisco startup, is building a Large Physics foundation Model to learn physics from sensory data. We seek data engineers to own datasets end-to-end—from source discovery to training-ready pipelines and access controls—ensuring data quality for multimodal physical data.

You will design scalable pipelines using Spark, Ray, and Beam, develop QA checks, and collaborate with researchers to verify that datasets translate into model performance.