Staff Data Engineer - End-to-End Ingestion & Quality

Causal Labs; San Francisco (CA); On-site; USD 120,000 - 170,000; Full time

Job summary

Causal Labs is building a Large Physics foundation Model to understand causality in physical systems, starting with weather.

We seek data engineers who will own datasets end-to-end, from discovery to training-ready pipelines, emphasizing data quality and scalable infrastructure.

Join a mission-driven team with deep AI and physics experience, collaborating with researchers and vendors to ensure data translates into model improvements.

Qualifications

• Experience building large-scale data pipelines, QA systems, or evaluation workflows.
• Detail-oriented with ability to identify subtle data inconsistencies and issues that could affect quality.
• Ability to learn unfamiliar domains quickly and read complex vendor/documentation.

Responsibilities

• Own every dataset end-to-end—from discovery and access to ingestion and training-ready data.
• Research and source new modalities of multimodal physical data and secure access through partnerships/vendors.
• Build petabyte-scale data pipelines (e.g. Apache Spark) for standardized data across batch and streaming.
• Develop quality metrics to measure coverage, correctness, and consistency across sources.
• Design automated QA checks and own the verdicts they produce.
• Write technical requirements and provide feedback to external data vendors and partners.
• Collaborate with researchers to validate that datasets translate into model performance.

Skills

Large-scale data pipelines

Data quality assurance

Research collaboration

Independent execution

Tools

Apache Spark

Beam

Ray

Job description

Causal Labs is building a Large Physics foundation Model to understand causality in physical systems, starting with weather.

We seek data engineers who will own datasets end-to-end, from discovery to training-ready pipelines, emphasizing data quality and scalable infrastructure.

Join a mission-driven team with deep AI and physics experience, collaborating with researchers and vendors to ensure data translates into model improvements.