Senior ML Inference Engineer — Scalable GPU Backend

Cavendish Professionals; Mountain View (CA); On-site; GBP 90,000 - 150,000; Full time

Job summary

Cavendish Professionals partners with an AI tech company to hire a Senior Machine Learning Engineer focused on production inference infrastructure. The role targets experienced inference or systems engineers who optimise GPU performance, model serving, and distributed systems at scale.

Responsibilities include building backend infra, improving throughput and latency, applying acceleration techniques, and scaling multi-GPU/multi-node setups.

Qualifications

• 5+ years of experience as an ML inference or systems engineer, with senior-level ownership.
• Hands-on experience with vLLM or TensorRT-LLM for inference optimisation.
• Experience building backend or ML systems and serving models in production.
• Strong systems programming and GPU optimisation using C++, CUDA, Rust or performant Python.
• Experience scaling distributed systems with Kubernetes, Ray, custom load balancing or multi-GPU/multi-node inference.
• Ability to explain performance trade-offs and the impact of engineering decisions.

Responsibilities

• Build and improve backend infrastructure for serving machine learning models in production.
• Optimise inference performance using vLLM or TensorRT-LLM.
• Improve throughput, latency and GPU utilisation across model-serving workloads.
• Apply model acceleration techniques such as quantisation, speculative decoding, caching, continuous batching and paged attention.
• Scale distributed inference infrastructure across multiple GPUs and nodes.
• Identify performance bottlenecks and improve reliability under high concurrency.
• Make architectural decisions across systems software, model-serving frameworks and infrastructure.

Skills

ML inference

Systems engineering

GPU optimisation

Performance tuning

Distributed systems

Model serving

Education

PhD in CS/Physics/Math or related (nice to have)

Tools

vLLM

TensorRT-LLM

Kubernetes

Ray

multi-GPU/multi-node inference

Job description

Cavendish Professionals partners with an AI tech company to hire a Senior Machine Learning Engineer focused on production inference infrastructure. The role targets experienced inference or systems engineers who optimise GPU performance, model serving, and distributed systems at scale.

Responsibilities include building backend infra, improving throughput and latency, applying acceleration techniques, and scaling multi-GPU/multi-node setups.