Staff AI Inference Performance Engineer (GenAI)

Google; Town of Montana (WI); On-site; USD 207,000 - 300,000; Full time

Benefits offered by this job

Bonus target

Equity

Job summary

Google DeepMind is hiring a Staff Software Engineer to optimize AI inference performance at scale. You will analyze the entire inference stack, implement optimization techniques, and drive systemic improvements to maximize throughput and reduce cost per inference.

You will collaborate with ML and systems teams on a range of AI agents and research prototypes. The role emphasizes end-to-end performance engineering, profiling workloads, and building instrumentation to guide capacity and latency

Qualifications

• Bachelor's degree or equivalent practical experience.
• 8 years of software development experience.
• Experience with Python and C++, including navigating and debugging serving codebases.
• Experience with AI model execution constraints, throughput-latency tradeoffs, memory bandwidth, and modern serving architectures.

Responsibilities

• Analyze and optimize AI inference workloads to increase throughput-per-GPU and reduce latency.
• Design and implement inference optimization techniques.
• Investigate and resolve complex model inference bottlenecks across the stack.
• Model latency-to-cost impacts and translate insights into production signals.
• Develop tools and metrics to track compute spend across the fleet.

Skills

Python

C++

Serving codebases

Performance optimization

Education

Bachelor's degree in CS/CE/EE or related field

Tools

vLLM

TensorRT-LLM

SGLang

Dynamo

Job description

Google DeepMind is hiring a Staff Software Engineer to optimize AI inference performance at scale. You will analyze the entire inference stack, implement optimization techniques, and drive systemic improvements to maximize throughput and reduce cost per inference.

You will collaborate with ML and systems teams on a range of AI agents and research prototypes. The role emphasizes end-to-end performance engineering, profiling workloads, and building instrumentation to guide capacity and latency