Senior TPM, AI Data & Evaluation Programs

Clera; San Francisco (CA); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

Visa sponsorship

Relocation support

Job summary

Clera in San Francisco seeks a Technical Program Manager to own complex data and evaluation programs for AI research teams and internal partners. You will structure ambiguous requests, define milestones, and build reliable processes to improve LLM and agent evaluations.

You will lead cross-functional programs from scoping to production, turning evolving requests into clear specifications and ensuring quality. The role involves managing vendors and enhancing data pipelines and workflows.

Qualifications

• Own complex AI/ML data and evaluation programs for research teams and partners.
• Lead cross-functional programs from scoping to production and retrospectives.
• Turn evolving requests into clear specs with milestones and acceptance criteria.
• Assess dataset quality with quantitative metrics and qualitative review.
• Manage external vendors across the delivery lifecycle and scale pipelines.

Responsibilities

• Lead data and evaluation programs from scoping and requirements gathering through production, quality assurance, delivery, and retrospectives.
• Turn evolving technical requests into clear specifications with milestones, owners, dependencies, and acceptance criteria.
• Assess and improve data quality using quantitative metrics and qualitative review, including task coverage, difficulty, and reward distributions.
• Manage external vendors across the delivery lifecycle and scale delivery pipelines.
• Improve workflows, documentation, automation, and cross-functional handoffs for data production.
• Work with research, platform, and marketplace teams to turn operational learnings into product improvements.

Skills

Program management

Vendor management

Data quality

Quantitative judgment

RLHF familiarity

Job description

Clera in San Francisco seeks a Technical Program Manager to own complex data and evaluation programs for AI research teams and internal partners. You will structure ambiguous requests, define milestones, and build reliable processes to improve LLM and agent evaluations.

You will lead cross-functional programs from scoping to production, turning evolving requests into clear specifications and ensuring quality. The role involves managing vendors and enhancing data pipelines and workflows.