Human Data Operations Lead — AI Model Training

Reflection; New York (NY); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

Top-tier compensation

Stock options

Health & wellness

Meals

23 days paid parental leave

Unlimited PTO (US)

Visa sponsorship

Team off-sites

Job summary

Reflection is seeking a Human Data Operations Manager to own data pipelines that improve AI models. You will partner with researchers, source data from vendors, and shape contributor workflows, data quality frameworks, and success metrics.

The role blends program ownership with hands-on design of tasks and processes. The ideal candidate has 4+ years delivering complex programs, understands LLM concepts, and is proficient in SQL and Python for analytics and automation.

Qualifications

• Minimum 4+ years delivering a portfolio of complex programs for research or data teams.
• Strong understanding of LLM training, evaluation, and prompt design or related AI research concepts.
• Proficiency in SQL for querying datasets and building dashboards; Python for data analysis and workflow automation.
• Experience with annotation, expert data collection, human feedback, or AI evaluation is a plus.

Responsibilities

• Collaborate with research to translate model capabilities into data acquisition strategies.
• Orchestrate providers: assess vendors, define task instructions, and ensure quality and cost targets.
• Build dashboards tracking throughput, turnaround time, acceptance rates, rework, and cost per task.
• Ensure alignment with evolving experiments and translate feedback into actionable sourcing changes.

Skills

Program management

SQL

Python

Data operations

Data annotation

Tools

Tableau

Airflow

Job description

Reflection is seeking a Human Data Operations Manager to own data pipelines that improve AI models. You will partner with researchers, source data from vendors, and shape contributor workflows, data quality frameworks, and success metrics.

The role blends program ownership with hands-on design of tasks and processes. The ideal candidate has 4+ years delivering complex programs, understands LLM concepts, and is proficient in SQL and Python for analytics and automation.