AI Evaluation Lead: Novel Testing Methodologies

Google; Washington (District of Columbia); On-site; USD 171,000 - 248,000; Full time

Benefits offered by this job

Health insurance

401(k) with company match

Paid time off 20 days

Sick time 40 hours

Maternity leave 28-30 weeks

Baby bonding leave 18 weeks

Paid holidays 13 days

Job summary

Google seeks a Novel AI Lead Methodologist to shape cutting-edge evaluation methodologies for emerging AI products. You will design evaluations for high-stakes bets, partner with engineering to build prototyping infrastructure, and drive safety mitigations across teams.

Many responsibilities involve bridging theory and execution, developing scalable testing approaches, and mentoring analysts while collaborating with data science and engineering to scale insights across Google’s ecosystem.

Qualifications

• Bachelor's degree or equivalent practical experience.
• 10 years of experience in AI testing or research, data analytics, data science, or a related field.

Responsibilities

• Drive the methodological frontier of model evaluation; develop data-driven testing methods.
• Define testing and safety standards; perform analyses to develop mitigations.
• Lead and influence cross-functional teams to implement safety initiatives; advise leadership.

Skills

Data analysis

Experiment design

Stakeholder influence

Research methods

Education

Bachelor's degree or equivalent practical experience

Master's degree or PhD

Tools

Python

SQL

Job description

Google seeks a Novel AI Lead Methodologist to shape cutting-edge evaluation methodologies for emerging AI products. You will design evaluations for high-stakes bets, partner with engineering to build prototyping infrastructure, and drive safety mitigations across teams.

Many responsibilities involve bridging theory and execution, developing scalable testing approaches, and mentoring analysts while collaborating with data science and engineering to scale insights across Google’s ecosystem.