Senior TPM – AI Data Labeling & Model Evaluation at Scale

Roblox; California (MO); On-site; USD 226,000 - 277,000; Full time

Job summary

Roblox is seeking a Senior Technical Program Manager for HITL Operations to own the end-to-end lifecycle of AI data labeling and model evaluation. You will scale data operations, manage a distributed contractor workforce, and drive platform evolution toward AI-assisted workflows.

You will partner with ML Engineers, Data Scientists, and Product Managers to translate model needs into executable data programs, define ground truth, and monitor quality metrics across the organization.

Qualifications

• 5+ years of experience in Technical Program Management, Data Operations, or Operations Management within an AI/ML or data-intensive environment.
• Hands-on data labeling, annotation, or model evaluation with scalable quality control.
• Proven experience managing external vendor relationships and distributed contractor workforces.
• Strong understanding of the ML lifecycle and role of human-labeled data in training and evaluating models.
• Proficiency in SQL and Python for querying datasets and building operational dashboards.
• Excellent written and verbal communication to present data concepts to researchers and business stakeholders.

Responsibilities

• Lead data programs end-to-end across labeling and model evaluation workflows for Roblox's AI teams.
• Define and maintain ground truth with labeling guidelines and quality frameworks for diverse data types.
• Manage a multi-vendor, distributed contractor workforce and forecast budgets.
• Drive the shift to AI-assisted workflows with LLM-based pre-labeling and evaluation.
• Collaborate with ML Engineers, Data Scientists, and Product Managers to translate needs into executable data programs.
• Define and track SLAs, surface risks, and drive continuous process improvement.
• Shape platform and tooling strategy, including vendor pilots and internal platform improvements.

Skills

TPM experience

Data Operations

SQL

Python

Budget management

Vendor management

Tools

Cursor

Claude

Gemini

Label Studio

Scale AI

Job description

Roblox is seeking a Senior Technical Program Manager for HITL Operations to own the end-to-end lifecycle of AI data labeling and model evaluation. You will scale data operations, manage a distributed contractor workforce, and drive platform evolution toward AI-assisted workflows.

You will partner with ML Engineers, Data Scientists, and Product Managers to translate model needs into executable data programs, define ground truth, and monitor quality metrics across the organization.