Staff Software Engineer, AI Evaluation & Quality

Google; Mountain View (CA); On-site; USD 207,000 - 300,000; Full time

Job summary

Google is hiring an Evaluation and Quality Engineer on the Search Evaluation Infrastructure team in Mountain View. You will pioneer Evaluation-Driven Development to bring objective, metrics-based quality to generative AI features across Search Verticals and International.

Expect to work across teams and impact AI feature readiness from day one. You will maximize fine-tuned evaluation model quality, streamline upstream signals, and drive AutoRater alignment.

Qualifications

• Minimum degree or equivalent practical experience, demonstrated in-depth software development.
• 8 years of hands-on software development experience in complex systems.
• Experience integrating generative AI tools or LLM interfaces into real workflows.
• Experience training or fine-tuning ML models, including evaluation setups.
• Ability to design and implement evaluation metrics for ML models.

Responsibilities

• Maximize evaluation quality and align AutoRater with cross-team signals.
• Lead core AutoRater design and scale data contributions across requirements.
• Align evaluation metrics with user growth and feedback signals for readiness.
• Build scalable automated rating infra and high-quality rubrics for metrics.
• Shift evaluations left, implementing metrics-based expertise for critical dimensions.

Skills

Bachelor’s degree or equivalent

8 years software development

GenAI / LLM integration experience

ML model training / fine-tuning

ML evaluation metrics frameworks

Education

Bachelor’s degree or equivalent practical experience

Job description

Google is hiring an Evaluation and Quality Engineer on the Search Evaluation Infrastructure team in Mountain View. You will pioneer Evaluation-Driven Development to bring objective, metrics-based quality to generative AI features across Search Verticals and International.

Expect to work across teams and impact AI feature readiness from day one. You will maximize fine-tuned evaluation model quality, streamline upstream signals, and drive AutoRater alignment.