Senior LLM Inference Engineer - Scale Secure Healthcare AI

Academic Key; Paris; Hybrid; EUR 90,000 - 140,000; Full time

Job summary

Doctolib is seeking a Senior Inference Engineer to power its AI-driven healthcare products. You will own the self-hosted LLM serving platform, deploy and tune open-weight models, and scale inference across GPU fleets with Ray Serve / Anyscale and Kubernetes-based orchestration.

Responsibilities include managing the model registry, benchmarking, and cost-control while ensuring security and compliance in healthcare data handling across Europe.

Qualifications

• 5+ years of software or computer engineering experience with production LLM serving.
• Hands-on with at least one LLM serving engine (vLLM, SGLang, TensorRT-LLM, TGI, or Triton).
• Proficiency in Python and Terraform; understanding transformer inference mechanics.

Responsibilities

• Build shared LLM serving foundations with MLOps engineers; define serving stack and deployment automation.
• Own model registry, versioning, validation, and promotion to production with quality benchmarks.
• Optimize inference performance, operation, and cost across GPU fleets; ensure reliability and observability.
• Maintain security and compliance in data residency and auditability across environments.

Skills

Python

English fluency

Communication

Tools

Terraform

Kubernetes

Ray Serve / Anyscale

vLLM

TensorRT-LLM

Triton Inference Server

MLflow

Datadog

KServe

Job description

Doctolib is seeking a Senior Inference Engineer to power its AI-driven healthcare products. You will own the self-hosted LLM serving platform, deploy and tune open-weight models, and scale inference across GPU fleets with Ray Serve / Anyscale and Kubernetes-based orchestration.

Responsibilities include managing the model registry, benchmarking, and cost-control while ensuring security and compliance in healthcare data handling across Europe.