AI Data Quality Engineer for Post-Training & Evaluation

Sierra Ventures; San Francisco (CA); Presencial; USD 190 000 - 230 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Stock options

Health & wellness

Meals provided in office

Parental leave

Resumo da oferta

Reflection is a research lab focused on making intelligence open and accessible for all. As a Data Team member, you will ensure the data used to train our models meets high quality standards and directly shape model performance on critical capabilities such as agentic tool use and long-horizon reasoning.

You will collaborate with world-class researchers, design scalable QA methods, and build reusable data pipelines that reliably deliver quality data to post-training teams for model training and

Qualificações

• Proficiency in Python and building ML/LLM workflows.
• Experience with large datasets and automated evaluation.
• Familiarity with how LLMs work and how models are trained/evaluated.
• Excellent communication across teams.

Responsabilidades

• Own upstream data quality for post-training and evaluation by analyzing expert datasets and defining quality signals.
• Collaborate with research/post-training teams to translate requirements into measurable signals and provide feedback to vendors.
• Design, validate, and scale automated QA methods (LLM-as-a-Judge) for large campaigns.
• Build reusable QA pipelines delivering high-quality data for model training/evaluation.
• Monitor data quality over time and iterate on standards and processes.

Conhecimentos

Python

ML workflows

Large datasets

LLM basics

Communication

Descrição da oferta de emprego

Reflection is a research lab focused on making intelligence open and accessible for all. As a Data Team member, you will ensure the data used to train our models meets high quality standards and directly shape model performance on critical capabilities such as agentic tool use and long-horizon reasoning.

You will collaborate with world-class researchers, design scalable QA methods, and build reusable data pipelines that reliably deliver quality data to post-training teams for model training and