Product Manager, AI Evals & Data Quality

Exa; San Francisco (CA); On-site; USD 140,000 - 210,000; Full time

Benefits offered by this job

Premium healthcare

Fertility benefits

16 weeks parental leave

Wellness stipend

Job summary

Exa, an applied AI lab, seeks a Product Manager for the Evals team in San Francisco to ensure evaluation data is comprehensive and high quality. You will partner with research, product, GTM, and engineering to shape data loops that improve search models.

You will manage external vendors, define metrics, and decide what data to acquire, influencing how we measure “good search” at scale for a world-class information system.

Qualifications

• Experience partnering with researchers, product and engineering to build evaluation datasets.
• Strong writing and communication skills with human-centered approach.
• Ability to manage external vendors and data-acquisition projects.

Responsibilities

• Define metrics and success criteria for evaluation data suites.
• Build localized eval feedback and signals to improve search quality.
• Collaborate with ML researchers, data engines, infra teams and product.

Skills

Vendor management

Written communication

Ownership

Numerical literacy

Education

CS/EE degree

Job description

Exa, an applied AI lab, seeks a Product Manager for the Evals team in San Francisco to ensure evaluation data is comprehensive and high quality. You will partner with research, product, GTM, and engineering to shape data loops that improve search models.

You will manage external vendors, define metrics, and decide what data to acquire, influencing how we measure “good search” at scale for a world-class information system.