Lead AI Evaluation & Testing Methodologist

Google; Kirkland (WA); On-site; USD 171,000 - 247,000; Full time

Benefits offered by this job

Health insurance

401(k) with company match

Paid time off

Sick time

Maternity leave

Baby bonding leave

Holidays

Job summary

Google’s Novel Testing team in the US seeks a senior researcher to lead emergent AI testing methodologies and evaluation frameworks. You will design high-impact experiments and partner with engineering to scale insights across products.

You will collaborate with data science and engineering to build prototype testing approaches, translate findings into scalable infrastructure, and mentor analysts while advancing safety practices.

Qualifications

• Bachelor's degree or equivalent practical experience.
• 10 years of experience in AI testing or related field.
• Experience building prototypes for AI testing with engineering teams.
• Experience in designing and conducting experiments or quantitative research in technology/AI context.

Responsibilities

• Drive the methodological frontier of model evaluation with data-driven methodologies.
• Define testing and safety standards with cross-functional partners; drive insights for mitigations.
• Lead and influence cross-functional teams to implement safety initiatives and advise leadership.
• Represent Google's AI safety efforts in external forums and collaborations; mentor analysts.
• Work with sensitive content and may engage with challenging material.

Skills

AI testing

Data analytics

Experiment design

Quantitative research

Cross-functional collaboration

Education

Bachelor's degree or equivalent practical experience

Master's degree or PhD in relevant field

Tools

SQL

Python

Job description

Google’s Novel Testing team in the US seeks a senior researcher to lead emergent AI testing methodologies and evaluation frameworks. You will design high-impact experiments and partner with engineering to scale insights across products.

You will collaborate with data science and engineering to build prototype testing approaches, translate findings into scalable infrastructure, and mentor analysts while advancing safety practices.