Pre-Training Data Quality Engineer (LLM Pipelines)

Sierra Ventures; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Top-tier compensation

Stock options

Health & wellness benefits

Meals in office

Paid parental leave

Unlimited vacation days

Visa sponsorship

Team offsites

Job summary

Reflection is a research lab focused on making intelligence open and accessible for everyone to use, customize, and build on. As a member of the Data Team, you will ensure the data used to train our models meets a high bar for quality, reliability, and downstream impact, directly shaping model performance.

You will own upstream data quality for LLM pre-training, translate quality requirements into measurable signals, and design automated QA methods to scale across large campaigns.

Qualifications

• Proficiency in Python and building ML/LLM workflows.
• Experience building data pipelines and QA systems.
• Strong ability to debug and write scalable code.
• Excellent communication across teams to convey complex concepts.

Responsibilities

• Own upstream data quality for LLM pre-training across languages and modalities.
• Translate requirements into measurable quality signals and provide feedback to data vendors.
• Design, validate, and scale automated QA methods to measure data quality.
• Build reusable QA pipelines delivering high-quality data to pre-training teams.
• Monitor and report data quality over time and drive process improvements.

Skills

Python

ML/LLM workflows

Data pipelines

Communication

Job description

Reflection is a research lab focused on making intelligence open and accessible for everyone to use, customize, and build on. As a member of the Data Team, you will ensure the data used to train our models meets a high bar for quality, reliability, and downstream impact, directly shaping model performance.

You will own upstream data quality for LLM pre-training, translate quality requirements into measurable signals, and design automated QA methods to scale across large campaigns.