Senior Machine Learning Engineer

Cavendish Professionals; Mountain View (CA); On-site; GBP 90,000 - 150,000; Full time

Job summary

Cavendish Professionals partners with an AI tech company to hire a Senior Machine Learning Engineer focused on production inference infrastructure. The role targets experienced inference or systems engineers who optimise GPU performance, model serving, and distributed systems at scale.

Responsibilities include building backend infra, improving throughput and latency, applying acceleration techniques, and scaling multi-GPU/multi-node setups.

Qualifications

• 5+ years of experience as an ML inference or systems engineer, with senior-level ownership.
• Hands-on experience with vLLM or TensorRT-LLM for inference optimisation.
• Experience building backend or ML systems and serving models in production.
• Strong systems programming and GPU optimisation using C++, CUDA, Rust or performant Python.
• Experience scaling distributed systems with Kubernetes, Ray, custom load balancing or multi-GPU/multi-node inference.
• Ability to explain performance trade-offs and the impact of engineering decisions.

Responsibilities

• Build and improve backend infrastructure for serving machine learning models in production.
• Optimise inference performance using vLLM or TensorRT-LLM.
• Improve throughput, latency and GPU utilisation across model-serving workloads.
• Apply model acceleration techniques such as quantisation, speculative decoding, caching, continuous batching and paged attention.
• Scale distributed inference infrastructure across multiple GPUs and nodes.
• Identify performance bottlenecks and improve reliability under high concurrency.
• Make architectural decisions across systems software, model-serving frameworks and infrastructure.

Skills

ML inference

Systems engineering

GPU optimisation

Performance tuning

Distributed systems

Model serving

Education

PhD in CS/Physics/Math or related (nice to have)

Tools

vLLM

TensorRT-LLM

Kubernetes

Ray

multi-GPU/multi-node inference

Job description

We’re partnering with an AI technology company hiring a Senior Machine Learning Engineer to build, optimise and scale production inference infrastructure.

This role suits an experienced inference or systems engineer who enjoys solving challenges around GPU performance, model serving and distributed systems. The focus is on making models run efficiently and reliably at scale.

What you’ll do

• Build and improve backend infrastructure for serving machine learning models in production.
• Optimise inference performance using vLLM or TensorRT-LLM.
• Improve throughput, latency and GPU utilisation across model-serving workloads.
• Apply model acceleration techniques such as quantisation, speculative decoding, caching, continuous batching and paged attention.
• Scale distributed inference infrastructure across multiple GPUs and nodes.
• Identify performance bottlenecks and improve reliability under high concurrency.
• Make architectural decisions across systems software, model-serving frameworks and infrastructure.

What we’re looking for

• 5+ years of experience as an ML inference or systems engineer, with senior-level technical ownership.
• Hands‑on experience with vLLM or TensorRT-LLM for inference optimisation.
• Experience building backend or ML systems and serving models in production.
• Strong systems programming and GPU optimisation skills, using C++, CUDA, Rust or performance-optimised Python.
• Practical experience with model acceleration techniques, such as quantisation, distillation, speculative decoding or attention and batching optimisations.
• Experience scaling distributed systems using technologies such as Kubernetes, Ray, custom load balancing or multi-GPU/multi-node inference.
• Ability to explain performance trade-offs and the impact of your engineering decisions.

Nice to have

• Open-source contributions to inference engines or relevant systems projects.
• Public technical writing demonstrating your expertise.
• Evidence of growing technical responsibility and career progression.
• A PhD in Computer Science, Physics, Mathematics or a related discipline, or equivalent practical systems experience.
• Ability to commute to Mountain View, California.