Senior LLM Inference Engineer — Hybrid, GPU-scale

Doctolib (France); Paris; Hybrid; EUR 90,000 - 150,000; Full time

Benefits offered by this job

Health insurance

Vacation days (25) and RTT

Mental health support and coaching

Relocation support for international移

Job summary

Doctolib is seeking a Senior Inference Engineer to lead the ML Platform (TIGER) in Paris. You will own a self-hosted LLM serving platform, deploying and tuning models like Gemma, Mistral, Llama, Qwen, and fine-tuned variants on Ray Serve/Anyscale and Kubernetes.

You’ll drive performance, reliability, security, and cost efficiency across Europe’s healthcare AI stack. Your role combines MLOps, model registry governance, production reliability, and observability with Datadog dashboards and audits

Qualifications

• 5+ years of software or computer engineering experience with LLM serving in production.
• Proficient in Python and Terraform; strong understanding of transformer inference mechanics.
• Experience running GPU workloads on Kubernetes or Ray/Anyscale; data-driven benchmarking mindset.

Responsibilities

• Build and operate shared LLM serving foundations with MLOps engineers.
• Own model registry, versioning, validation, and production promotion.
• Tune serving engines to reduce latency and cost per token with batching and quantization.
• Define SLOs, run load tests, and implement safe rollouts and autoscaling.
• Create Datadog dashboards and cost reporting to monitor GPU utilization and inference spend.

Skills

Python

Terraform

Kubernetes

MLOps

LLM serving

GPU scheduling

Datadog

Observability

Tools

Ray Serve

Anyscale

Kubernetes

vLLM

Triton

TensorRT-LLM

SGLang

KServe

CUDA

Datadog

Job description

Doctolib is seeking a Senior Inference Engineer to lead the ML Platform (TIGER) in Paris. You will own a self-hosted LLM serving platform, deploying and tuning models like Gemma, Mistral, Llama, Qwen, and fine-tuned variants on Ray Serve/Anyscale and Kubernetes.

You’ll drive performance, reliability, security, and cost efficiency across Europe’s healthcare AI stack. Your role combines MLOps, model registry governance, production reliability, and observability with Datadog dashboards and audits