Data Acquisition & Operations Lead for Open Models

Reflection; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Top-tier compensation

Stock options

Health & wellness

Meals provided

Parental leave

Unlimited PTO

Visa sponsorship

Team events

Job summary

Reflection is a research lab focused on making intelligence open and accessible. We are seeking a Human Data Operations Manager to own data acquisition pipelines, collaborate with researchers, and source from vendors to improve model usefulness and performance.

You will design task instructions, establish data quality frameworks, track metrics, and ensure alignment with evolving research priorities. Experience with annotation or human feedback is a plus.

Qualifications

• 4+ years delivering a portfolio of complex programs across changing research needs.
• Strong understanding of LLM training, evaluation, and prompt design.

Responsibilities

• Translate research plans into data acquisition strategies.
• Define and manage vendor capabilities, instructions, and quality criteria.
• Build dashboards to track throughput, turnaround time, acceptance rates, and cost per task.
• Collaborate with researchers to align data sourcing with model development goals.

Skills

LLM concepts

Written communication

Stakeholder management

Bias to action

Tools

SQL

Python

APIs

Job description

Reflection is a research lab focused on making intelligence open and accessible. We are seeking a Human Data Operations Manager to own data acquisition pipelines, collaborate with researchers, and source from vendors to improve model usefulness and performance.

You will design task instructions, establish data quality frameworks, track metrics, and ensure alignment with evolving research priorities. Experience with annotation or human feedback is a plus.