ML Data Operations Lead — Labeling & Quality

Coalition; United States; Remote; USD 134,400 - 200,000; Full time

Benefits offered by this job

100% medical, dental and vision

Flexible PTO

Annual home office stipend

Wellness programs

Advancement opportunities

Job summary

Coalition seeks a data operations lead to manage labeling quality and methodology. You will own annotation guidelines, gold standards, IAA targets, and audits, and oversee the Label Studio platform configuration and integration.

You will coordinate with ML and product teams to structure labeling tasks and supervise outsourced labeling vendors to meet quality and throughput targets. The role reports to the Chief Product Officer, sits at the intersection of product, ML, and operations, and holds

Qualifications

• Experience designing annotation guidelines and gold standards.
• Knowledge of IAA targets, calibration workflows, and audit sampling.
• Proven track record managing outsourced labeling vendors.

Responsibilities

• Define annotation guidelines, taxonomies, and edge-case protocols.
• Own Label Studio configuration, workflows, and integration points.
• Collaborate with ML engineers, data scientists, and product managers.
• Manage external labeling vendors and ensure SLAs and quality targets.
• Define metrics and drive improvements in labeling quality and throughput.

Skills

Labeling quality frameworks

IAA targets

Python and SQL

Cross-functional collaboration

ML data operations

Quality audits

Data labeling methodolog

Tools

Label Studio

Scale AI

Labelbox

Prodigy

Job description

Coalition seeks a data operations lead to manage labeling quality and methodology. You will own annotation guidelines, gold standards, IAA targets, and audits, and oversee the Label Studio platform configuration and integration.

You will coordinate with ML and product teams to structure labeling tasks and supervise outsourced labeling vendors to meet quality and throughput targets. The role reports to the Chief Product Officer, sits at the intersection of product, ML, and operations, and holds