Senior AI Inference Engineer — Scale Secure LLM Serving

Doctolib; Paris; Sur place; EUR 90 000 - 140 000; Plein temps

Résumé du poste

Doctolib in Paris is seeking a Senior Inference Engineer to join the ML Platform (TIGER) team. You will own the self-hosted LLM serving platform powering our AI-driven healthcare products, from Consultation Assistant to Health Companion, and scale models across our GPU fleet.

You will build the shared foundations for AI features, deploy and tune open-weight models, ensure security and compliance, and drive performance and cost efficiency across Europe.

Qualifications

• 5+ years of software or computer engineering experience with hands-on production experience with at least one LLM serving engine.
• Proficient in Python and Terraform; solid understanding of transformer inference mechanics.
• Experience running GPU workloads on Kubernetes or a managed Ray / Anyscale platform; performance engineering mindset.
• Comfortable owning production reliability: observability, incident response, and on-call.
• Fluent in English.

Responsabilités

• Build the shared LLM serving foundations with MLOps engineers - define the serving stack, GPU capacity planning, and deployment automation for open-weight models on Ray Serve / Anyscale, moving toward Kubernetes-based orchestration.
• Own the model registry and promotion path - establish versioning, validation, and production promotion; benchmark performance and output quality; maintain the curated model catalog with the Agents Platform team.
• Optimize inference performance and efficiency - operate and tune serving engines to reduce latency and cost across the GPU fleet, using batching, caching, quantization, and parallelism.
• Own production reliability and disaster recovery - define SLOs, run load tests before launches, implement safe rollouts, rollbacks, and autoscaling, and participate in on-call.
• Make serving performance and costs measurable - build Datadog dashboards, alerting, and cost reporting to track latency, GPU utilization, and inference spend.
• Keep the inference stack secure, compliant, and evolving - collaborate with Security and Privacy on data residency and auditability; assess new models and engines and GPU generations.

Connaissances

Python

Terraform

Kubernetes

Performance engineering

Observability

English fluency

Outils

Ray Serve

Anyscale

Triton Inference Server

vLLM

TensorRT-LLM

MLflow

KServe

NVIDIA Dynamo

Description du poste

Doctolib in Paris is seeking a Senior Inference Engineer to join the ML Platform (TIGER) team. You will own the self-hosted LLM serving platform powering our AI-driven healthcare products, from Consultation Assistant to Health Companion, and scale models across our GPU fleet.

You will build the shared foundations for AI features, deploy and tune open-weight models, ensure security and compliance, and drive performance and cost efficiency across Europe.