Staff Software Engineer, GenAI Inference Performance

Google LLC; Mountain View (CA); On-site; USD 207,000 - 300,000; Full time

Job summary

DeepMind is seeking a Staff Software Engineer for Inference Performance Optimization, GenAI. You will work on measuring intelligence of prototypes, building systems for agent testing, and developing test problems in physics simulators, while visualizing results and leading leaderboards.

You will push AI model execution at scale, analyze the full inference stack, maximize hardware throughput, reduce cost-to-serve, and enable data-driven capacity and latency tradeoffs across teams.

Qualifications

• Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, Applied Mathematics, or a related technical field, or equivalent practical experience.
• 8 years of experience in software development.
• Experience in Python and C++, including navigating, debugging, and modifying serving codebases.

Responsibilities

• Analyze and optimize AI inference workloads across the application, model, and distributed fleet infrastructure layers to methodically increase throughput-per-GPU and reduce latency.
• Design and implement inference optimization techniques.
• Investigate and resolve complex model inference performance bottlenecks across the stack.
• Model the latency-to-cost impacts of system variables (such as batch-sizing and utilization goals) and translate these insights into actionable signals that drive production systems.
• Develop investigative tools and metrics (e.g., compute/FLOPs funnels) that track where compute is spent across the fleet.

Skills

Python

C++

Education

Bachelor's degree

Job description

DeepMind is seeking a Staff Software Engineer for Inference Performance Optimization, GenAI. You will work on measuring intelligence of prototypes, building systems for agent testing, and developing test problems in physics simulators, while visualizing results and leading leaderboards.

You will push AI model execution at scale, analyze the full inference stack, maximize hardware throughput, reduce cost-to-serve, and enable data-driven capacity and latency tradeoffs across teams.