Staff Technical Product Manager, Evals

Ladders; United States; Remote; USD 200,000 - 350,000; Full time

Benefits offered by this job

Unlimited AI token budget

Flexible vacation

Healthcare coverage

Vision coverage

Dental coverage

Daily meals and snacks

Job summary

Ladders is seeking a Staff Technical Product Manager, Evals to lead evaluation strategy for an AI-driven coding product. You will collaborate with engineering to translate goals into technical requirements and shape how user data is used safely to guide product decisions.

This cross-functional role emphasizes privacy-aware data practices, strong product judgment, and the ability to operate in a fast-moving AI environment while working remotely in the United States.

Qualifications

• 5+ years of product management with technical evals.
• Strong software development, system design, and API design knowledge.
• Experience creating and managing eval methodologies.
• Understanding of UX design and product sense.
• Familiarity with AI context and engineering best practices.

Responsibilities

• Drive development of AI coding agent to improve cost, speed, and accuracy.
• Manage offline and online eval sets for product performance.
• Use extensive user data to inform eval creation.
• Ensure customer data privacy through anonymization practices.
• Collaborate with engineering on complex implementation challenges.

Skills

Product management

Eval methodologies

API design

AI context

Privacy-aware data

Cross-functional leadership

TypeScript

React

Job description

For our client, we are seeking a Staff Technical Product Manager, Evals to join the team of a leader in the Enterprise Technology space. This role will own the evaluation strategy for an AI-driven coding product, with a focus on improving cost, speed, and accuracy across both offline and online measurement frameworks. The position partners closely with engineering to translate product goals into practical technical requirements, while also shaping how user data is used safely and effectively to guide product decisions. It is a highly cross-functional role that blends technical product leadership, evaluation methodology, privacy-aware data practices, and strong product judgment in a fast-moving AI environment. Location: Remote. Compensation: $200,000 – $350,000 annually.

Responsibilities

• Drive the development of the company's AI coding agent to improve cost, speed, and accuracy
• Manage both offline and online eval sets for product performance
• Use extensive user data to inform eval creation
• Ensure customer data privacy through effective anonymization practices
• Collaborate with engineering teams on complex implementation challenges

Qualifications

• 5-7 years of experience in product management, particularly with technical evals
• Strong technical skills in software development, system design, and API design
• Proven experience in creating and managing eval methodologies
• Deep understanding of user experience design and product sense
• Familiarity with AI context and harness engineering best practices
• Proficient in core technologies like TypeScript and React
• Ability to leverage AI to enhance workflow efficiency

Benefits

• Unlimited AI token budget
• Flexible vacation policy allowing you to take what you need
• Comprehensive healthcare, vision, and dental coverage
• Daily meals and snacks provided in the office
• Opportunity to achieve your best work in a supportive environment

Our client is an equal opportunity employer. We encourage you to apply even if you don’t meet every qualification—your background could be exactly what this team needs.