Inference Foundation Engineer – Scalable LLM Serving

Mistral.ai; Paris; Hybrid; EUR 90,000 - 150,000; Full time

Benefits offered by this job

Healthcare coverage

Parental leave

Retirement plans

Relocation support

Wellness programs

Meal allowances

Transportation allowances

Job summary

Mistral.ai is seeking a skilled ML/LLM Infrastructure Engineer for a hybrid role in Paris. You will work on the core inference engine and orchestration, optimize serving at scale, and enable frontier model training with low latency and high availability.

Your stack includes vLLM, SGLang, TensorRT-LLM, CUDA, NCCL, and Kubernetes, with Python and PyTorch in production. You will contribute to scalable, high-performance AI systems while collaborating across teams to deliver robust serving

Qualifications

• Experience building and running ML/LLM services at scale with latency and availability targets.

Responsibilities

• Develop and fix the core of the inference stack — engine and orchestrator — including feature selection, configuration, and tuning for maximum performance at scale
• Own the release process for the serving stack: validated, regression-free releases through automated performance gates and progressive rollout
• Drive improvements and fixes upstream when the open-source engine is the right place for them
• Optimize serving efficiency across the fleet — driving down pod startup time, tackling cold-cache regressions on scale-up, smarter caching and offloading
• Optimize and maintain the optimal serving topology — overlap communication and transfers with computation, ensure optimal placement, connectivity, and routing
• Build the serving infrastructure that powers RL and post-training for our frontier models
• Optimize inference performance across the full spectrum of our workloads

Skills

ML/LLM services at scale

Latency targets

Availability targets

Python for systems tooling

PyTorch

CUDA/NCCL

Kubernetes

Tools

vLLM

SGLang

TensorRT-LLM

CUDA

Kubernetes

PyTorch

Job description

Mistral.ai is seeking a skilled ML/LLM Infrastructure Engineer for a hybrid role in Paris. You will work on the core inference engine and orchestration, optimize serving at scale, and enable frontier model training with low latency and high availability.

Your stack includes vLLM, SGLang, TensorRT-LLM, CUDA, NCCL, and Kubernetes, with Python and PyTorch in production. You will contribute to scalable, high-performance AI systems while collaborating across teams to deliver robust serving