Senior AI Inference Engineer — Scale LLM Serving

Doctolib GmbH; Paris; Hybrid; EUR 90,000 - 140,000; Full time

Benefits offered by this job

Health insurance

Vacation days

Mental health support

Lunch vouchers

Transport subsidy

Relocation support

DoctoGrowth program

Flexible work policy

Job summary

Doctolib is seeking a Senior Inference Engineer to own the self-hosted LLM serving platform in the ML Platform team. You will deploy, tune, and scale open-weight models across our GPU fleet, with a focus on safety, efficiency, and cost.

You will collaborate with Security and Privacy to meet data residency and auditability requirements, while advancing production readiness. You will drive performance benchmarking, build observability dashboards in Datadog, and own reliability responsibilities

Qualifications

• 5+ years of software or computer engineering experience with LLM serving in production.
• Proficient in Python and Terraform with transformer inference knowledge.
• Experience running GPU workloads on Kubernetes or Ray/Anyscale.
• Strong observability, incident response, and on-call familiarity.
• Excellent communication and collaborative mindset, English fluency.

Responsibilities

• Build shared LLM serving foundations with MLOps engineers and deployment automation.
• Own model registry, versioning, validation, and production promotion processes.
• Optimize inference performance with batching, caching, and quantization strategies.
• Ensure production reliability, SLOs, load testing, and safe rollouts.
• Develop dashboards and cost reporting to monitor latency and GPU usage.

Skills

Python

Terraform

Kubernetes

Ray Serve

MLOps

Datadog

GPU workloads

Model registry

Tools

vLLM

TensorRT-LLM

Triton

SGLang

Kubernetes

Job description

Doctolib is seeking a Senior Inference Engineer to own the self-hosted LLM serving platform in the ML Platform team. You will deploy, tune, and scale open-weight models across our GPU fleet, with a focus on safety, efficiency, and cost.

You will collaborate with Security and Privacy to meet data residency and auditability requirements, while advancing production readiness. You will drive performance benchmarking, build observability dashboards in Datadog, and own reliability responsibilities