Data Ingestion Engineering Lead

Reflection; New York (NY); On-site; USD 140,000 - 230,000; Full time

Benefits offered by this job

Top-tier compensation

Stock options

Health & wellness

Meals provided

Parental leave

Unlimited PTO (US)

Visa sponsorship

Job summary

Reflection is seeking a Data Ingestion Lead to head the team building the ingestion layer for our frontier models. You will mentor data ingestion engineers, guide architecture across crawling, extraction, and corpus storage, and work with pre-training and data partnerships teams to ensure reliable, auditable data at scale.

You’ll stay hands-on while shaping strategy, driving scalable pipelines, and delivering timely data to support model training.

Qualifications

• Experience building, mentoring, and growing data or infrastructure engineering teams while staying technically hands-on.
• Deep experience building web-scale data acquisition or ingestion systems, with ownership of production-grade pipelines at multi-TB to PB scale.
• Strong coding ability and credibility to earn technical trust.
• Deep expertise in web crawling & acquisition, large-scale extraction & ingestion pipelines, or data lakes / corpus storage & delivery; able to learn the rest.
• Fluency with modern data toolkits: distributed compute (Ray, Beam, Spark), orchestration (Airflow, Prefect), formats (Parquet, JSONL, WARC), and data-lake architectures.
• Familiarity with how LLMs are trained and evaluated; able to design experiments and use proxy quality signals to guide improvements.
• Ability to guide strategy and execution across concurrent data campaigns, partnering with research, vendors, and partnerships/legal on acquisition.

Responsibilities

• Build, mentor, and grow a high-performing data ingestion team.
• Provide front-line leadership across the ingestion stack: web crawling, extraction pipelines, and data lakes.
• Stay hands-on to contribute as an individual contributor when needed.
• Prioritize work and run data acquisition campaigns in a fast-paced environment.
• Guide technical and architectural decisions emphasizing scalability, reliability, and cost-aware design.
• Collaborate with research and partnerships teams to tie ingestion outcomes to model impact.

Skills

Mentoring teams

Web-scale ingestion

Distributed compute

Airflow/Prefect

Parquet/JSONL/WARC

Tools

Ray

Beam

Spark

Airflow

Prefect

Parquet

JSONL

WARC

Job description

Reflection is seeking a Data Ingestion Lead to head the team building the ingestion layer for our frontier models. You will mentor data ingestion engineers, guide architecture across crawling, extraction, and corpus storage, and work with pre-training and data partnerships teams to ensure reliable, auditable data at scale.

You’ll stay hands-on while shaping strategy, driving scalable pipelines, and delivering timely data to support model training.