Senior AI Evaluation Methodologist

Socket.dev; Washington (District of Columbia); On-site; USD 171,000 - 247,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

Google is seeking a senior researcher in AI testing and evaluation to design novel testing methodologies for emergent AI, partnering with data science and engineering teams to build scalable infrastructure and experiments.

The role focuses on advancing evaluation frameworks, mitigating model risks, and informing product safety decisions; it involves collaboration across DeepMind and Google’s Data Science teams, with exposure to evolving AI capabilities.

Qualifications

• Bachelor's degree or equivalent practical experience.
• 5+ years of data analysis for AI testing or related field.

Responsibilities

• Drive the methodological frontier of model evaluation with data-driven frameworks for testing.
• Define testing and safety standards with cross-functional teams and develop mitigations.
• Lead cross-functional teams to implement safety initiatives and advise leadership on complex safety issues.
• Represent Google's AI safety efforts in external forums and mentor analysts on adversarial techniques.
• Work with sensitive content and may encounter graphic or upsetting topics.

Skills

Data analysis

AI testing

Experimentation

Quantitative research

Cross-functional collaboration

Education

Bachelor's degree or equivalent practical experience

Master's degree or PhD (preferred)

Tools

Python

SQL

Job description

Google is seeking a senior researcher in AI testing and evaluation to design novel testing methodologies for emergent AI, partnering with data science and engineering teams to build scalable infrastructure and experiments.

The role focuses on advancing evaluation frameworks, mitigating model risks, and informing product safety decisions; it involves collaboration across DeepMind and Google’s Data Science teams, with exposure to evolving AI capabilities.