Senior AI Data Ingestion Engineer

Sierra Ventures; San Francisco (CA); Hybrid; USD 180,000 - 280,000; Full time

Benefits offered by this job

Top-tier compensation

Stock options

Health & wellness coverage

Meals in office

Parental leave (22 weeks)

Unlimited vacation (US)

Visa sponsorship

Job summary

Reflection is seeking a data engineer to build and operate ingestion systems for pre‑training at scale. You will own web crawling, extraction, normalization, and delivery pipelines, working with researchers to align data with model goals.

You will run experiments, analyze data quality, and ensure observability and reliability across multi‑TB to PB datasets in a hybrid research‑engineering team.

Qualifications

• Experience building web crawling, data ingestion, or large‑scale data acquisition systems using Ray, Beam, Spark, or similar technologies.
• Familiarity with how LLMs are trained and evaluated, and an intuition for what makes data useful for training
• Comfortable working with very large datasets (multi‑TB to PB scale) and building systems that are observable, testable, and maintainable
• Comfortable designing experiments and using data to guide system improvements
• Excellent communication skills. You can explain system behavior. You consider and communicate tradeoffs clearly

Responsibilities

• Build and operate ingestion systems for pre‑training at large scale
• Run experiments to evaluate crawling strategies and ingestion tradeoffs
• Analyze ingested data to identify gaps and redundancies
• Build scalable ingestion pipelines for large data campaigns
• Review code and debug production issues, improving ingestion infrastructure

Skills

Web crawling

Data ingestion

LLM knowledge

Experiment design

Communication skills

Tools

Ray

Beam

Spark

Job description

Reflection is seeking a data engineer to build and operate ingestion systems for pre‑training at scale. You will own web crawling, extraction, normalization, and delivery pipelines, working with researchers to align data with model goals.

You will run experiments, analyze data quality, and ensure observability and reliability across multi‑TB to PB datasets in a hybrid research‑engineering team.