Staff AI Quality Engineer — LLM & Model Evaluation

Google; New York (NY); On-site; USD 207,000 - 300,000; Full time

Job summary

Google is seeking a Staff Software Engineer in Applied AI, Model Quality in New York, NY. You will lead the development of evaluation systems for model quality, working across Gemini and GemPix components to drive measurable improvements on a 2026 roadmap.

The role requires deep expertise in ML infrastructure, model deployment and evaluation, and experience integrating generative AI tools. In-person interviewing is typically required, and compensation includes a high seven-figure scale with

Qualifications

• Bachelor's degree or equivalent practical experience.
• 8 years of experience in software development.
• 5 years of experience with Speech/Audio, reinforcement learning, ML infrastructure, or related field.
• 5 years of experience with ML design and ML infrastructure (model deployment, evaluation, data processing, debugging, fine tuning).
• 5 years of experience testing, launching software products, and 3 years of experience with software design and architecture.
• Experience integrating generative AI tools or LLM interfaces into workflows.

Responsibilities

• Build both human-powered and LLM-powered automated evaluation systems to assess model performance.
• Establish clear metrics to measure grounding, coherence, safety, and helpfulness.
• Utilize platforms and tools to run evaluations across models and datasets.
• Provide actionable insights from evaluations to improve model quality, in collaboration with research and cross-functional teams.
• Create tools and systems that make the evaluation process more efficient.

Skills

Software development

ML infrastructure

Speech/Audio

Reinforcement learning

Model evaluation

Generative AI / LLM integration

ML design

Education

Bachelor's degree or equivalent

Master's or PhD in Engineering/CS

Job description

Google is seeking a Staff Software Engineer in Applied AI, Model Quality in New York, NY. You will lead the development of evaluation systems for model quality, working across Gemini and GemPix components to drive measurable improvements on a 2026 roadmap.

The role requires deep expertise in ML infrastructure, model deployment and evaluation, and experience integrating generative AI tools. In-person interviewing is typically required, and compensation includes a high seven-figure scale with