LLM Inference Research Engineer — Latency & Throughput

Lindus Health Limited; Greater London; Hybrid; GBP 70,000 - 110,000; Full time

Job summary

Mistral is seeking a Research Engineer on the Latency & Throughput team to push the frontier of LLM inference. You will research and prototype techniques to reduce latency and increase throughput, from speculative decoding to KV compression, then ship proven methods into our serving stack.

You will work between inference engineers and research scientists, producing proofs of concept that bridge academia and production. Expect collaboration across teams and potential co-training scenarios.

Qualifications

• Strong ML background with research in LLMs and inference techniques.
• Experience designing and running experiments to validate hypotheses.
• Familiarity with state-of-the-art LLM inference techniques and benchmarks.
• Track record of rigorous research with clear conclusions from results.
• Ability to work across research and production to ship impactful methods.

Responsibilities

• Research and prototype inference techniques and model architectures.
• Design and run experiments to validate ideas and demonstrate latency/throughput gains.
• Collaborate with Inference Foundation to integrate techniques into serving stack.
• Coordinate with pretraining/RL teams when co-training or retraining is needed.
• Build tooling and validation platforms for AI-driven inference research.

Skills

ML research

Python proficiency

Experiment design

Communication

Self-motivated

Education

PhD not required

Tools

Python

PyTorch

JAX

CUDA

vLLM

Job description

Mistral is seeking a Research Engineer on the Latency & Throughput team to push the frontier of LLM inference. You will research and prototype techniques to reduce latency and increase throughput, from speculative decoding to KV compression, then ship proven methods into our serving stack.

You will work between inference engineers and research scientists, producing proofs of concept that bridge academia and production. Expect collaboration across teams and potential co-training scenarios.