Senior AI Evaluation Leaderboard Product Manager

Socket.dev; New York (NY); On-site; USD 206,000 - 257,000; Full time

Job summary

Scale is seeking a Senior AI Product Manager to own and scale Scale’s SEAL Leaderboard portfolio from strategy to execution. You will align AI PM, ML researchers, engineers, and GTM teams to turn model evaluations into trusted industry benchmarks that influence development and purchasing decisions.

Ideal candidates bring 5+ years in product management or related roles, strong technical fluency in ML systems, and proven ability to ship end-to-end products with governance, infrastructure, and

Qualifications

• 5+ years of experience in product management, technical program management, consulting, or customer-facing technical roles.
• Strong technical fluency, including familiarity with machine learning systems, AI model evaluation, benchmarking, or data products.
• Experience building and scaling products that require coordination across engineering, operations, and business teams.
• Excellent stakeholder management and executive communication skills, with a demonstrated ability to drive alignment across cross-functional organizations.
• Strong analytical skills and the ability to translate ambiguous market and customer signals into clear product strategy.

Responsibilities

• Own the roadmap and strategy for Scale’s SEAL Leaderboard portfolio, defining priorities across benchmark development, leaderboard launches, infrastructure investments, and product expansion.
• Facilitate the Leaderboard Steering Committee and drive alignment across AI-PM, ML, Engineering, Operations, and GTM stakeholders.
• Evaluate, prioritize, and operationalize new leaderboard proposals, ensuring alignment with customer demand, market opportunities, and company strategy.
• Define and manage the end-to-end leaderboard product lifecycle, from ideation and benchmark design to launch, growth, maintenance, and sunset decisions.
• Partner with ML researchers and domain experts to develop trustworthy evaluation methodologies, benchmark specifications, and leaderboard scoring frameworks.
• Drive the roadmap for leaderboard infrastructure, automation, and operational tooling to improve scalability, reduce manual effort, and accelerate benchmark releases.
• Establish governance processes for benchmark quality, evaluation integrity, release management, auditability, and update cadence.
• Work directly with frontier AI labs and enterprise customers to understand evaluation needs, gather feedback, and influence future leaderboard investments.
• Track adoption, usage, customer impact, and business outcomes, using data-driven insights to guide roadmap decisions and resource allocation.
• Identify new evaluation categories, benchmark opportunities, and leaderboard products that strengthen Scale’s industry leadership and create new revenue opportunities.
• Collaborate closely with GTM teams to support customer engagements, thought leadership initiatives, product launches, and strategic partnerships.

Skills

Product management

Technical fluency

Stakeholder management

Executive communication

Analytical skills

AI collaboration

Entrepreneurial mindset

Bias for action

Job description

Scale is seeking a Senior AI Product Manager to own and scale Scale’s SEAL Leaderboard portfolio from strategy to execution. You will align AI PM, ML researchers, engineers, and GTM teams to turn model evaluations into trusted industry benchmarks that influence development and purchasing decisions.

Ideal candidates bring 5+ years in product management or related roles, strong technical fluency in ML systems, and proven ability to ship end-to-end products with governance, infrastructure, and