Data Quality Engineer for AI Model Pre-Training

Reflection AI Ltd; New York (NY); On-site; USD 140,000 - 200,000; Full time

Benefits offered by this job

Top-tier compensation

Stock options

Health & wellness

Meals in office

Parental leave

Unlimited PTO

Visa sponsorship

Team building

Job summary

Reflection AI Ltd is seeking a data-quality oriented engineer to ensure the data used for pre-training meets high standards. You will own upstream data quality across languages and modalities, translating requirements into measurable signals and feedback for data vendors.

Collaborating with researchers and pre-training teams, you will design and scale automated QA methods, build reusable pipelines, and monitor quality over campaigns to drive improvements in model performance.

Qualifications

• Proficiency in Python and scalable ML/LLM workflows.
• Experience with large datasets and automated evaluation or quality-checking systems.
• Understanding of how data quality impacts pre-training and model performance.
• Ability to design and validate automated quality checks (rule-based, statistical, or model-assisted).
• Excellent communication across teams and stakeholders.

Responsibilities

• Own upstream data quality for LLM pre-training across languages and modalities.
• Translate requirements into measurable quality signals and provide feedback to data vendors.
• Design, validate, and scale automated QA methods for data quality across campaigns.
• Build reusable QA pipelines delivering high-quality data to pre-training teams.
• Monitor and report data quality over time and drive iterative improvements.

Skills

Python

ML workflows

Data pipelines

QA systems

Communication

Job description

Reflection AI Ltd is seeking a data-quality oriented engineer to ensure the data used for pre-training meets high standards. You will own upstream data quality across languages and modalities, translating requirements into measurable signals and feedback for data vendors.

Collaborating with researchers and pre-training teams, you will design and scale automated QA methods, build reusable pipelines, and monitor quality over campaigns to drive improvements in model performance.