Member of Technical Staff — Data Ingestion & Quality

Kindredventures; San Francisco (CA); On-site; USD 120,000 - 180,000; Full time

Job summary

Kindredventures is building a Large Physics foundation Model and seeks a Data Engineer to own data end-to-end, from source discovery to ingestion pipelines ensuring training-ready quality.

You will build petabyte-scale pipelines, implement QA checks, and collaborate with researchers to translate datasets into model performance improvements. This role demands problem-solving, fast execution, and adaptability.

Qualifications

• Experience building large-scale data pipelines and QA systems.
• Ability to translate data quality into model performance.

Responsibilities

• Own data end-to-end: source discovery, access, ingestion pipelines, training-ready data.

Skills

Large-scale data pipelines

QA systems

Data ingestion

Multimodal data

Vendor collaboration

Tools

Apache Spark

Ray

Beam

Job description

Our mission is general causal intelligence; AI that is capable of (1) predicting the future and (2) identifying the actions to alter it.

To achieve this breakthrough, we are building a Large Physics foundation Model (LPM) because physical systems, unlike text or images, are governed by verifiable cause and effect. We believe that scaling on physics will enable an understanding of causality required to predict and control physical systems, starting with weather.

Our founding team has built and deployed AI against the physical world in robotics, drug discovery, and particle physics at institutions like DeepMind, Waymo, Cruise, Insitro, Nabla Bio, and CERN.

We look for data engineers who are excited to tackle unsolved problems. Data is critical to any ML model but is especially consequential for our thesis to learn physics from sensory observations. The vast majority of meaningful progress in AI comes not from new architectures, but from training on data that is carefully curated with specific characteristics, quality, and scale.

Responsibilities

Your mission is to own every dataset end to end — from discovering the source and securing access, to writing the pipelines that ingest it, to guaranteeing it enters training clean, standardized, and correct.

• Research and source new modalities of multimodal physical data (e.g. sparse sensors, point clouds, hyperspectral imagery, radar), and secure access through partnerships, vendors, and public archives
• Build petabyte-scale data pipelines (e.g. Apache Spark) that ingest each source into our storage in standardized, training-ready form, across both batch and streaming — including the orchestration, storage, and monitoring they need where shared platform infrastructure doesn\'t yet exist
• Develop quality metrics that measure coverage, correctness, and consistency across sources — and catch the subtle inconsistencies (sensor bias, drift, processing artifacts) that silently degrade models
• Design and implement automated QA checks that continuously measure and monitor data quality over time, and own the verdicts they produce
• Write technical requirements and provide actionable feedback to external data vendors and partners
• Collaborate with researchers to validate that new and improved datasets translate into model performance

What we\'re looking for

We value a relentless approach to problem-solving, rapid execution, and the ability to quickly learn in unfamiliar domains.

• Demonstrated experience building large-scale data pipelines, QA systems, or evaluation workflows (e.g. Spark, Ray, Beam)
• Detail-oriented in identifying subtle data inconsistencies and issues that could affect quality, with the ability to understand how quality impacts model performance
• Comfortable going deep on unfamiliar source material — reading format specifications, sensor documentation, and vendor manuals to get ingestion exactly right
• Experience working with external data vendors and partners, from technical evaluation to ongoing feedback
• Owns deliverables end-to-end, from collecting and translating requirements to autonomously driving execution