Staff Software Engineer, AI Evaluation & Quality for Search

Google Inc.; Mountain View (CA); On-site; USD 207,000 - 300,000; Full time

Job summary

Google in Mountain View, CA seeks a Staff Software Engineer for the Search Evaluation and Quality Team. The role focuses on Evaluation-Driven Development to embed objective, metrics-based quality checks early in product definition, supporting generative AI features across Search Verticals and International teams.

You will drive methods to maximize fine-tuned evaluation model quality, streamline upstream signals, and align metrics with long-term user growth while collaborating across engineering

Qualifications

• Bachelor's degree or equivalent practical experience.
• 8 years of experience in software development.
• Experience integrating generative AI tools or LLM interfaces into workflows.
• Experience with ML model training or fine-tuning (e.g., LLMs, generative AI).
• Experience developing evaluation frameworks or metrics for machine learning models.

Responsibilities

• Maximize fine-tuned evaluation model quality, streamline data upstream signals, and establish precise AutoRater alignment in collaboration with horizontal teams.
• Lead the core vertical AutoRater hillclimb and manage continuous vertical data contributions while scaling evaluation capabilities across different requirements.
• Align evaluation metrics with long-term user growth metrics and direct user feedback signals to bridge the functionality readiness gap before feature launches.
• Develop scalable automated rating infrastructure, high-quality evaluation rubrics, and continuous tracking systems to enable objective, metrics-based expertise.
• Shift evaluations left to eliminate subjective quality assessments and implement metrics-based expertise starting with critical dimensions like "helpfulness."

Skills

Outcome ownership

GenAI tools

ML model training

Education

Bachelor's degree or equivalent practical experience

Tools

AutoRater

Job description

Google in Mountain View, CA seeks a Staff Software Engineer for the Search Evaluation and Quality Team. The role focuses on Evaluation-Driven Development to embed objective, metrics-based quality checks early in product definition, supporting generative AI features across Search Verticals and International teams.

You will drive methods to maximize fine-tuned evaluation model quality, streamline upstream signals, and align metrics with long-term user growth while collaborating across engineering