Senior HITL Ops TPM — AI Data Labeling & Evaluation

The available sources do not contain information about the company name for rounx.com.; San Mateo (CA); On-site; USD 226,000 - 277,000; Full time

Job summary

Roblox in San Mateo, CA, is seeking a Senior Technical Program Manager for HITL Operations to own end-to-end data labeling and model evaluation workflows across AI teams, managing a distributed contractor workforce and multi-million budget.

You will define ground truth, build labeling guidelines, drive AI-assisted workflows, partner with ML Engineers and Product Managers, and own KPIs like throughput and data quality, shaping the next-gen labeling platform.

Qualifications

• 5+ years of experience in Technical Program Management, Data Operations, or Operations Management within an AI/ML or data-intensive environment.
• Hands-on experience with data labeling, annotation, or model evaluation—designing annotation schemas and labeling guidelines at scale.
• Proven experience managing external vendor relationships and distributed contractor workforces, including budgeting and quality oversight.

Responsibilities

• Lead data programs end-to-end across labeling and model evaluation workflows for Roblox's AI teams.
• Define and maintain ground truth with labeling guidelines and quality frameworks for diverse data types.
• Manage a multi-vendor, distributed workforce with SLAs and budget forecasting.
• Drive AI-assisted workflows, evaluating LLM-based pre-labeling and automated QA routing.
• Collaborate with ML Engineers, Data Scientists, and Product Managers to translate needs into executable data programs.
• Own operational excellence by tracking SLAs, data quality, and cost per label; drive continuous improvements.

Skills

Technical Program Management

Data Operations

Vendor management

SQL

Python

Communication

Cross-functional collaboration

Ambiguity management

Tools

Label Studio

Scale AI

Snorkel AI

Cursor

Claude

Gemini

Job description

Roblox in San Mateo, CA, is seeking a Senior Technical Program Manager for HITL Operations to own end-to-end data labeling and model evaluation workflows across AI teams, managing a distributed contractor workforce and multi-million budget.

You will define ground truth, build labeling guidelines, drive AI-assisted workflows, partner with ML Engineers and Product Managers, and own KPIs like throughput and data quality, shaping the next-gen labeling platform.