Staff Software Engineer, AI Evaluation Platform

Google DeepMind; Mountain View (CA); On-site; USD 207,000 - 300,000; Full time

Job summary

Google DeepMind is seeking a Software Engineer to help build the GenAI Eval Platform, focusing on scalable evaluation backbones for frontier AI models and agentic workflows. You will collaborate with ML and neuroscience researchers to shape model capability, safety, and readiness for global deployment.

You will design, implement, and test evaluation systems using cutting-edge frameworks, while mentoring teams toward rigorous engineering practices in a mission-driven environment.

Qualifications

• Bachelor's degree in Computer Science, Software Engineering, Mathematics, or a related STEM field, or equivalent practical experience.
• 5 years of experience coding in one or more general purpose languages (e.g., Python, Java, C or C++).
• 5 years of experience designing and launching large-scale production software products or foundation AI systems.
• 4 years of experience in software development.
• 3 years of experience leading software design and system architecture for high-throughput distributed systems, foundation model runtimes, or client applications.

Responsibilities

• Design and optimize distributed evaluation execution engines to orchestrate large volumes of inference steps across TPU and GCU pools with high throughput and low latency.
• Build foundational abstractions for evaluating complex LLM agent loops, tool use, and automated LLM-as-a-judge rating systems.
• Design robust error classification, automated retry policies, and observability dashboards to maintain strict SLOs for evaluation pipeline success rates.
• Partner closely with GDM research scientists and data science teams to anticipate frontier model evaluation requirements and translate them into scalable infrastructure solutions.
• Mentor fellow engineers, set high standards for code quality (Python in Google3), and advocate rigorous testing and system design practices.

Skills

Distributed systems

High-throughput architectures

Leadership

Python

Java

C/C++

Education

Bachelor's degree in CS or related field

Master's or PhD preferred

Tools

PyTorch

JAX

Triton

vLLM

Job description

Google DeepMind is seeking a Software Engineer to help build the GenAI Eval Platform, focusing on scalable evaluation backbones for frontier AI models and agentic workflows. You will collaborate with ML and neuroscience researchers to shape model capability, safety, and readiness for global deployment.

You will design, implement, and test evaluation systems using cutting-edge frameworks, while mentoring teams toward rigorous engineering practices in a mission-driven environment.