ML Inference Engineer - LLM Deployment & Serving

Google DeepMind; Mountain View (CA); Hybrid; USD 230,000 - 290,000; Full time

Job summary

Google DeepMind seeks a Software Engineer to advance large-scale ML deployment and research-to-production workflows. You will collaborate with ML researchers to optimize model serving on GPUs/TPUs, build efficient test infrastructure, and deploy agents and LLMs on Google's production stack.

You'll work across IC and TL paths, contributing to performance tuning, architecture decisions, and end-to-end lifecycle tooling for research-to-production pipelines in a mission-driven team.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 8 years of experience in software development.
• 2 years of experience in deploying and maintaining machine learning models in a live production environment.
• Experience in profiling, configuring, or executing ML workloads directly on hardware accelerators (e.g., GPU or TPU).
• Experience designing, building, or optimizing model serving infrastructure or inference backends.

Responsibilities

• Collaborate closely with Research teams to understand next generation modeling approaches, ensuring they are designed and implemented with production considerations in mind.
• Work with infrastructure teams to deliver serving infrastructure that is designed for maximum efficiency and performance, addressing bottlenecks in speed, scale, and quality.
• Identify opportunities to automate tasks, eliminate redundancies, build performant tests, and improve the overall velocity of model releases.
• Gain a deep understanding of serving frameworks, pre-processing pipelines, caching mechanisms, and other relevant technologies.
• Leverage roofline analysis, hardware-level profiling, and systems analysis to identify and eliminate performance bottlenecks across ML frameworks, compilers (XLA), custom kernels (Pallas), and serving infrastructure on hardware accelerators (TPUs/GPUs).

Skills

Software development

ML deployment

Hardware accelerators

Model serving

Education

Bachelor’s degree

Tools

JAX

PyTorch

CUDA/OpenCL

Job description

Google DeepMind seeks a Software Engineer to advance large-scale ML deployment and research-to-production workflows. You will collaborate with ML researchers to optimize model serving on GPUs/TPUs, build efficient test infrastructure, and deploy agents and LLMs on Google's production stack.

You'll work across IC and TL paths, contributing to performance tuning, architecture decisions, and end-to-end lifecycle tooling for research-to-production pipelines in a mission-driven team.