Staff AI Data & Benchmark Systems Engineer

Google LLC; United States; On-site; GBP 120,000 - 180,000; Full time

Job summary

Google LLC is seeking Staff Software Engineers to advance evaluation and training data pipelines across AI data and Gemini/Dart initiatives. You will design and implement scalable benchmarks and automate quality gates, while guiding architecture across cross-functional teams.

Role emphasizes leadership in a fast-paced, multi-project environment with deep expertise in distributed systems, AI, and large-scale data processing. In-person interviews are typical.

Qualifications

• 8 years of programming experience in C++, Java, Python, Kotlin or Go.
• 5 years of testing and launching software products.
• 3 years of software design and architecture.
• Experience integrating generative AI tools or LLM interfaces into workflows.
• Bachelor's degree or equivalent practical experience.

Responsibilities

• Research and implement novel techniques for accelerated evaluation creation, including automated task synthesis, programmatic verifiers, and model-assisted rubric induction.
• Architect pipelines to mine execution logs, agent traces, and runtime artifacts into high-fidelity evals and training data that continuously self-improve.
• Design and build robust benchmarks evaluating complex reasoning across code, math, and specialized domains; optimize for climbability, discriminability, and zero contamination.
• Build and scale platform software that automates quality gates—such as AutoRaters and automated loss-pattern root-cause analyzers.
• Provide technical direction and architecture across AI Data and partners; align distributed engineering teams.

Skills

Programming languages

Software testing

Software design

AI/LLM integration

Leadership

Cross-functional work

Education

Bachelor's degree or equivalent

Master's or PhD in Engineering/CS

Job description

Google LLC is seeking Staff Software Engineers to advance evaluation and training data pipelines across AI data and Gemini/Dart initiatives. You will design and implement scalable benchmarks and automate quality gates, while guiding architecture across cross-functional teams.

Role emphasizes leadership in a fast-paced, multi-project environment with deep expertise in distributed systems, AI, and large-scale data processing. In-person interviews are typical.