Senior AI Evaluation Leaderboard Product Manager

United States Digital Space LLC; New York (NY); On-site; USD 206,000 - 257,000; Full time

Benefits offered by this job

Equity

Health/dental/vision coverage

Retirement benefits

Learning & development stipend

Generous PTO

Job summary

United States Digital Space LLC is seeking a Senior AI Product Manager to own and scale the SEAL Leaderboard portfolio. You will define strategy, roadmaps, and ensure operational excellence for evaluation products, benchmarks, and leaderboards.

You will collaborate with AI PM, ML researchers, engineering, operations, and GTM teams to turn model evaluations into trusted industry benchmarks driving decisions.

Qualifications

• 5+ years of experience in product management, technical program management, consulting, or customer-facing technical roles.
• Strong technical fluency with machine learning systems, AI model evaluation, benchmarking, or data products.
• Experience coordinating across engineering, operations, and business teams.
• Excellent stakeholder management and executive communication skills.

Responsibilities

• Own the roadmap and strategy for Scale’s SEAL Leaderboard portfolio, defining priorities across benchmark development, leaderboard launches, infrastructure investments, and product expansion.
• Facilitate the Leaderboard Steering Committee and drive alignment across AI-PM, ML, Engineering, Operations, and GTM stakeholders.
• Evaluate, prioritize, and operationalize new leaderboard proposals aligned with customer demand and market opportunities.
• Define and manage the end-to-end leaderboard product lifecycle from ideation to launch, growth, maintenance, and sunset decisions.
• Partner with ML researchers to develop trustworthy evaluation methodologies and leaderboard scoring frameworks.
• Drive the roadmap for infrastructure, automation, and tooling to improve scalability and accelerate releases.
• Establish governance for benchmark quality, evaluation integrity, release management, and auditability.
• Work with frontier AI labs and enterprise customers to understand needs and guide investments.
• Track adoption and business impact to inform roadmap decisions and resource allocation.
• Identify new evaluation categories and leaderboard opportunities to strengthen leadership and revenue.
• Collaborate with GTM teams on customer engagements and strategic partnerships.

Skills

Product management

Technical program management

Consulting

Customer-facing roles

Machine learning evaluation

Benchmarking

Job description

United States Digital Space LLC is seeking a Senior AI Product Manager to own and scale the SEAL Leaderboard portfolio. You will define strategy, roadmaps, and ensure operational excellence for evaluation products, benchmarks, and leaderboards.

You will collaborate with AI PM, ML researchers, engineering, operations, and GTM teams to turn model evaluations into trusted industry benchmarks driving decisions.