Senior ML Inference Engineer for LLM Serving

WeAreTechWomen; United Kingdom; On-site; GBP 120,000 - 190,000; Full time

Job summary

Google DeepMind seeks a Software Engineer to join the team responsible for bringing AI research to life, deploying large language models (LLMs) like Gemini onto production infrastructure and building robust, high-performance ML systems.

You will work across research and engineering to optimize and deploy LLMs in production, with opportunities across IC and TL tracks, across multiple teams focused on serving and inference dynamics.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 8 years of experience in software development.
• 2 years of experience in deploying and maintaining ML models in a live production environment.
• Experience in profiling, configuring, or executing ML workloads directly on hardware accelerators (e.g., GPU or TPU).
• Experience designing, building, or optimizing model serving infrastructure or inference backends.

Responsibilities

• Collaborate with Research teams to understand modeling approaches with production considerations.
• Work with infrastructure teams to deliver efficient serving infrastructure for maximum speed and scale.
• Identify opportunities to automate tasks, remove redundancies, and improve velocity of model releases.
• Gain deep understanding of serving frameworks, preprocessing pipelines, caching mechanisms, and related tech.
• Leverage hardware-level profiling and roofline analysis to optimize ML frameworks, compilers (XLA), and kernels (Pallas) on accelerators.

Skills

Software development experience

ML deployment in production

Profiling ML workloads on GPUs/TPUs

Model serving infrastructure

Experience with AI systems

Education

Bachelor’s degree or equivalent practical experience

Tools

CUDA

OpenCL

JAX

PyTorch

XLA

Pallas

GPU/TPU

Job description

Google DeepMind seeks a Software Engineer to join the team responsible for bringing AI research to life, deploying large language models (LLMs) like Gemini onto production infrastructure and building robust, high-performance ML systems.

You will work across research and engineering to optimize and deploy LLMs in production, with opportunities across IC and TL tracks, across multiple teams focused on serving and inference dynamics.