LLM Inference & Serving Engineer

Mistral AI; Greater London; Hybrid; GBP 120,000 - 180,000; Full time

Benefits offered by this job

Healthcare coverage

Relocation support

Wellness programs

Job summary

Mistral AI is seeking a senior engineer to own and optimize the inference stack, engine orchestration, and scalable serving for frontier AI models. You will work on production-grade ML/LLM services, capacity sizing, and training-serving pipelines to keep models responsive and cost-efficient.

The role emphasizes building high-performance serving systems, with hands-on experience in vLLM, SGLang, or TensorRT-LLM, and deep CUDA/NCCL knowledge. Hybrid work in London area is supported.

Qualifications

• Experience building and running ML/LLM services at scale with latency/availability targets.
• Hands-on experience with inference engines such as vLLM, SGLang, TensorRT-LLM or equivalents.
• Solid grasp of inference internals: prefill vs decode, KV-cache, batching, scheduling, parallelism strategies.
• Familiar with distributed and disaggregated serving architectures.

Responsibilities

• Develop and fix the core of the inference stack — engine and orchestrator — for max performance at scale.
• Own the release process for the serving stack with automated performance gates and progressive rollout.
• Drive improvements upstream when open-source engine is appropriate.
• Optimize serving efficiency across the fleet and topology for best throughput and latency.
• Build serving infra powering RL and post-training workloads.

Skills

ML at scale

Inference engines

Python

PyTorch

Kubernetes

CUDA/NCCL

GPU hardware

Networking

Distributed systems

Tools

vLLM

SGLang

TensorRT-LLM

Job description

Mistral AI is seeking a senior engineer to own and optimize the inference stack, engine orchestration, and scalable serving for frontier AI models. You will work on production-grade ML/LLM services, capacity sizing, and training-serving pipelines to keep models responsive and cost-efficient.

The role emphasizes building high-performance serving systems, with hands-on experience in vLLM, SGLang, or TensorRT-LLM, and deep CUDA/NCCL knowledge. Hybrid work in London area is supported.