Novel AI Lead Methodologist

Google; Atlanta (GA); Hybrid; USD 171,000 - 247,000; Full time

Benefits offered by this job

Health insurance

Retirement 401(k)

Paid time off

Sick time

Maternity leave

Baby bonding

Holiday pay

Job summary

Google is seeking an AI safety researcher within Trust and Safety to lead novel testing methodologies for emergent AI and to design evaluation frameworks for GenAI products. You will collaborate with data science and engineering teams to build scalable testing infrastructure and prototype experiments that identify systemic risks.

US-based candidates will enjoy a comprehensive benefits package, a competitive base salary, a 20% bonus target, equity, and paid time off with multiple US locations and

Qualifications

• Bachelor's degree or equivalent practical experience.
• 10 years of experience in AI testing or research, data analytics, data science, or a related field.
• Master's degree or PhD in relevant field preferred.
• 5 years of experience in data analysis for AI Testing with SQL or Python.
• Experience building prototypes and partnering with engineering teams.
• Experience designing experiments or quantitative research in AI contexts.
• Knowledge of AI systems, machine learning, and potential risks.
• Strong data-driven investigative skills with Python and SQL.

Responsibilities

• Drive methodological frontier of model evaluation with data-driven testing frameworks.
• Define testing and safety standards; develop model- and product-level mitigations.
• Lead cross-functional teams; advise executive leadership on safety issues.
• Represent Google's AI safety efforts in external forums and collaborations.
• Work with sensitive content; handle potentially upsetting topics.

Skills

Data analysis

Python

SQL

AI testing

Education

Bachelor's degree or equivalent practical experience

Master's degree or PhD in relevant field

Tools

Python

SQL

Job description

In accordance with Washington state law, we are highlighting our comprehensive benefits package, which is available to all eligible US based employees. Benefits for this role include:

• Health, dental, vision, life, disability insurance
• Retirement Benefits: 401(k) with company match
• Paid Time Off: 20 days of vacation per year, accruing at a rate of 6.15 hours per pay period for the first five years of employment
• Sick Time: 40 hours/year (increased to 69 hours/year for Seattle) including 5 discretionary sick days per instance
• Maternity Leave (Short-Term Disability + Baby Bonding): 28-30 weeks
• Baby Bonding Leave: 18 weeks
• Holidays: 13 paid days per year

Note: By applying to this position you will have an opportunity to share your preferred working location from the following: Washington D.C., DC, USA; Atlanta, GA, USA; Austin, TX, USA; Kirkland, WA, USA; Seattle, WA, USA.

Minimum qualifications

• Bachelor's degree or equivalent practical experience.
• 10 years of experience in AI testing or research, data analytics, data science, or a related field.

Preferred qualifications

• Master's degree or PhD in relevant field.
• 5 years of experience in data analysis for AI Testing with experience in SQL or Python.
• Experience building or partnering with engineering teams to build prototypes for AI testing.
• Experience in designing and conducting experiments or quantitative research, preferably in a technology or AI context.
• Experience in AI systems, machine learning, and their potential risks.
• Strong technical competency with a data-driven investigative approach to solve complex tests, including demonstrable proficiency in data manipulation, analysis, and automation using languages like Python and SQL.

About The Job

Novel Testing is a team within Trust and Safety specializing in complex testing, defining protocols and methodologies for assessing risk where best practices do not currently exist. We pioneer and scale testing programs, streamlining the launch of trustworthy, novel AI products.

Work spans from designing first-of-their-kind evaluations for Google’s most ambitious product bets—including autonomous agents, personalization, and the latest hardware— to developing new methodologies for assessing novel foundational model capabilities as they emerge.

Advancing in AI evaluation is central to this mission. To scale these methods, we partner closely with engineering teams to build the infrastructure and tools required for automated evaluation.

In this role, you will lead the development of novel testing methodologies for emergent AI, designing evaluation frameworks where established standards do not yet exist. You will address complex testing questions with creative experimentation, designing sophisticated prompt strategies and quantitative analyses to identify systemic risks and edge cases in GenAI products.

Bridging the gap between theory and execution, you will move quickly to build and prototype testing solutions that incorporate methodological best practices. You will then partner directly with data science and engineering teams to inform the development of novel testing approaches and automated infrastructure, ensuring your insights scale effectively across Google’s ecosystem. This position demands a researcher’s mindset—capable of deep qualitative and quantitative inquiry—paired with the technical agility to translate those findings into scalable, engineering prototypes.

Individual pay is determined by factors including job-related skills, experience, and relevant education or training.

US: $171000 - $247000 (USD) + 20% bonus target + equity + benefits

Responsibilities

Learn more about benefits at Google.

• Drive the methodological frontier of model evaluation. Partner with DeepMind and Data Science, developing novel, data-driven methodologies for structured and unstructured testing of emerging AI products. Move beyond standard benchmarks, designing sophisticated experimental frameworks, uncovering latent model behaviors and capabilities.
• Define testing and safety standards, working with cross-functional colleagues to ensure they are met. Perform analyses and drive insights to develop model-level and product-level safety mitigations.
• Lead and influence cross-functional teams to implement safety initiatives. Advise executive leadership on complex safety issues.
• Represent Google's AI safety efforts in external forums and collaborations, contributing to industry-wide best practices. Mentor analysts, fostering a culture of excellence, acting as a subject matter expert on adversarial techniques.
• Work with sensitive content or situations and may be exposed to graphic, controversial or upsetting topics or content.

Google is proud to be an equal opportunity workplace and is an affirmative action employer. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. See also Google's EEO Policy and EEO is the Law. If you have a disability or special need that requires accommodation, please let us know by completing our Accommodations for Applicants form.