Senior AI Evaluation & Leaderboard PM

Scale AI; San Francisco (CA); On-site; USD 206,000 - 257,000; Full time

Benefits offered by this job

Equity

Benefits

PTO

Commuter stipend

Job summary

Scale AI in San Francisco seeks a Senior AI Product Manager to own Scale’s SEAL Leaderboard portfolio, shaping strategy, roadmap, and governance of evaluation products and benchmarks. You will collaborate with AI PM, ML researchers, engineering, and GTM teams to transform model evaluations into trusted industry benchmarks that influence model development and purchasing decisions.

The ideal candidate brings strong product judgment, technical fluency, operational rigor, and customer-facing

Qualifications

• 5+ years of experience in product management, technical program management, consulting, or customer-facing technical roles.
• Strong technical fluency with ML systems, AI model evaluation, benchmarking, or data products.
• Experience coordinating across engineering, operations, and business teams.
• Excellent stakeholder management and executive communication with cross-functional alignment.
• Ability to translate ambiguous market signals into clear product strategy.

Responsibilities

• Own the roadmap and strategy for Scale’s SEAL Leaderboard portfolio, defining priorities across benchmark development, leaderboard launches, and infrastructure investments.
• Facilitate the Leaderboard Steering Committee and drive alignment across AI-PM, ML, Engineering, Operations, and GTM stakeholders.
• Evaluate and operationalize new leaderboard proposals, ensuring alignment with customer demand and market opportunities.
• Define end-to-end leaderboard product lifecycle from ideation to launch, growth, maintenance, and sunset decisions.
• Partner with ML researchers to develop evaluation methodologies, benchmark specs, and leaderboard scoring frameworks.
• Drive infrastructure, automation, and tooling to improve scalability and accelerate releases.
• Establish governance for benchmark quality, evaluation integrity, release management, and update cadence.
• Engage with frontier AI labs and enterprise customers to collect feedback and influence investments.
• Track adoption, usage, and business impact to guide roadmap and resource allocation.
• Identify new evaluation categories and revenue opportunities; collaborate with GTM for product launches.

Skills

Product management

Technical program management

Executive communication

Stakeholder management

Analytical skills

Job description

Scale AI in San Francisco seeks a Senior AI Product Manager to own Scale’s SEAL Leaderboard portfolio, shaping strategy, roadmap, and governance of evaluation products and benchmarks. You will collaborate with AI PM, ML researchers, engineering, and GTM teams to transform model evaluations into trusted industry benchmarks that influence model development and purchasing decisions.

The ideal candidate brings strong product judgment, technical fluency, operational rigor, and customer-facing