Senior LLM Serving Engineer – AI Infra & MLOps

Doctolib; Paris; Hybrid; EUR 90,000 - 130,000; Full time

Benefits offered by this job

Free health insurance

25 vacation days per year

Lunch vouchers

Relocation support

Transit subsidy

Employee growth program

Job summary

Doctolib in Paris, France, is seeking a Senior Inference Engineer to own the self-hosted LLM serving platform for healthcare AI products. You will deploy, tune, and scale open-weight models across GPU fleets, drive model registry, and optimize performance and cost while ensuring security and regulatory compliance.

You will collaborate with MLOps to define serving stacks, monitor latency, and build robust dashboards.

Qualifications

• 5+ years of software or computer engineering experience, with production exposure to an LLM serving engine.
• Proficient in Python and Terraform, strong understanding of transformer inference mechanics.
• Experience running GPU workloads on Kubernetes or Ray/Anyscale; performance-driven decision making.
• Comfortable owning production reliability: observability and incident response.

Responsibilities

• Build the shared LLM serving foundations with MLOps engineers and define serving stack and deployment automation for open-weight models on Ray Serve / Anyscale.
• Own the model registry and promotion path; benchmark performance and maintain the model catalog with the Agents Platform team.
• Optimize inference performance and efficiency by tuning serving engines and applying batching, caching, and quantization strategies.
• Own production reliability, disaster recovery, SLOs, load tests, and on-call involvement.
• Make serving performance and costs measurable with dashboards, alerting, and cost reporting.

Skills

Python

Terraform

Kubernetes

LLM serving engines

GPU workloads

Tools

Ray Serve

Anyscale

Docker

Terraform

Job description

Doctolib in Paris, France, is seeking a Senior Inference Engineer to own the self-hosted LLM serving platform for healthcare AI products. You will deploy, tune, and scale open-weight models across GPU fleets, drive model registry, and optimize performance and cost while ensuring security and regulatory compliance.

You will collaborate with MLOps to define serving stacks, monitor latency, and build robust dashboards.