Senior AI Inference Engineer - Hybrid, Open-Weight LLM

Doctolib (France); France; Híbrido; EUR 110.000 - 170.000; Tempo integral

Benefícios oferecidos por esta vaga

Health insurance

Vacation days

Mental health support

Relocation assistance

Flexible work policy

Resumo da vaga

Doctolib is hiring a Senior Inference Engineer to join the ML Platform (TIGER) team in AI & Machine Learning. You will own the self-hosted LLM serving platform, deploying and tuning open-weight models across a GPU fleet, and ensuring data remains within Doctolib infrastructure you control.

You will lead the model registry, optimize inference performance, and drive reliability with scalable deployments. The role requires strong Python and Terraform skills, Kubernetes or Ray/Anyscale

Qualificações

• 5+ years of software or AI infrastructure experience.
• Hands-on production experience with at least one LLM serving engine.
• Proficient in Python and Terraform.
• Experience running GPU workloads on Kubernetes or a managed Ray / Anyscale platform.
• Fluent in English.

Responsabilidades

• Build the shared LLM serving foundations with MLOps engineers — define the serving stack, GPU capacity planning, and deployment automation for open-weight models on Ray Serve / Anyscale, while shaping the evolution toward Kubernetes-based orchestration.
• Own the model registry and promotion path — establish clear processes for versioning, validating, and promoting models into production, benchmark performance and output quality under realistic workloads, and maintain the curated model catalog together with the Agents Platform team.
• Optimize inference performance and efficiency — operate and tune serving engines to reduce latency and cost per token through batching, caching, quantization, and parallelism strategies across our GPU fleet.
• Own production reliability and disaster recovery — define SLOs, run load tests before launches, implement safe rollouts, rollbacks, and autoscaling, and participate in on-call and incident response.
• Make serving performance and costs measurable — build Datadog dashboards, alerting, and cost reporting to track latency, GPU utilization, and inference spend, and identify opportunities to improve resource efficiency.
• Keep the inference stack secure, compliant, and evolving — work with Security and Privacy on network isolation, data residency, and auditability, while continuously assessing new models and GPU generations.

Conhecimentos

Python

Terraform

Kubernetes

LLM serving

English fluency

Ferramentas

Ray Serve

Anyscale

Triton Inference Server

TensorRT-LLM

Descrição da vaga

Doctolib is hiring a Senior Inference Engineer to join the ML Platform (TIGER) team in AI & Machine Learning. You will own the self-hosted LLM serving platform, deploying and tuning open-weight models across a GPU fleet, and ensuring data remains within Doctolib infrastructure you control.

You will lead the model registry, optimize inference performance, and drive reliability with scalable deployments. The role requires strong Python and Terraform skills, Kubernetes or Ray/Anyscale