Research Engineer: AI Inference & Latency

Mistral; Greater London; Hybrid; GBP 90,000 - 130,000; Full time

Job summary

Mistral is seeking a Research Engineer for the Latency & Throughput team to push the frontier of LLM inference. You will explore speculative decoding, quantization, and KV compression, turning research ideas into production-ready improvements within our vLLM-based serving stack.

You will work closely with inference engineers and researchers, running rigorous experiments, and contributing to design choices that balance latency gains with model quality.

Qualifications

• Strong ML background with LLM training concepts (architecture, tokenization, fine-tuning)
• Hands-on Python and PyTorch; able to write clean, experimental code
• Experience reading literature and designing controlled experiments
• Track record of hypothesis testing and drawing conclusions from results

Responsibilities

• Research and prototype inference techniques (speculative decoding, quantization, KV compression) to reduce latency
• Design experiments to validate ideas and demonstrate latency/throughput gains
• Collaborate with Inference Foundation to integrate techniques into the serving stack
• Coordinate with pretraining/RL teams for co-training or retraining when needed

Skills

Python & PyTorch

JAX familiarity

CUDA basics (GPU)

Research experience

Tools

vLLM serving stack

Experimentation tooling

Job description

Mistral is seeking a Research Engineer for the Latency & Throughput team to push the frontier of LLM inference. You will explore speculative decoding, quantization, and KV compression, turning research ideas into production-ready improvements within our vLLM-based serving stack.

You will work closely with inference engineers and researchers, running rigorous experiments, and contributing to design choices that balance latency gains with model quality.