Senior ML Inference & Serving Engineer (LLM Deploy)

Google DeepMind; Mountain View (CA); On-site; USD 207,000 - 300,000; Full time

Job summary

Google DeepMind in Mountain View, CA is seeking a Software Engineer to advance AI agents and deploy large language models on production infrastructure. You will collaborate with researchers and engineers to optimize serving frameworks, test new algorithms, and scale inference on GPUs/TPUs.

The role offers IC and TL opportunities across teams, with a strong emphasis on performance, reliability, and impact. The team culture emphasizes collaboration, experimentation, and measurable progress in AI

Qualifications

• Bachelor's degree or equivalent practical experience required.
• 8 years of software development experience.
• 2 years deploying and maintaining ML models in production.
• Experience with hardware accelerators (GPU/TPU).
• Experience designing or optimizing model serving infrastructure.

Responsibilities

• Collaborate with Research teams to design production-friendly modeling approaches.
• Work with infrastructure teams to deliver efficient serving infrastructure.
• Identify opportunities to automate tasks and improve model release velocity.
• Understand serving frameworks, pre-processing pipelines, caching, and related tech.
• Use profiling and systems analysis to eliminate bottlenecks across ML frameworks and kernels.

Skills

Bachelor's degree or equivalent

8 years experience in software dev

2 years deploying ML models in prod

Hardware accelerators (GPU/TPU) exp

Model serving infra design

Education

Bachelor's degree or equivalent practical experience

Tools

Serving infra development

ML frameworks exposure (JAX, PyTorch)

Job description

Google DeepMind in Mountain View, CA is seeking a Software Engineer to advance AI agents and deploy large language models on production infrastructure. You will collaborate with researchers and engineers to optimize serving frameworks, test new algorithms, and scale inference on GPUs/TPUs.

The role offers IC and TL opportunities across teams, with a strong emphasis on performance, reliability, and impact. The team culture emphasizes collaboration, experimentation, and measurable progress in AI