Senior LLM Infrastructure Engineer - Training & Inference

Datadog, Inc.

Paris

On-site

EUR 120,000 - 180,000

Full time

7 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

RSUs and ESPP
Conference attendance
Mentor program
Inclusive culture

Job summary

Datadog Paris is seeking a Senior AI Research Engineer to join the Frontier Foundations team, building systems for large-scale LLM training, reinforcement learning, and inference. You will design and operate a distributed runtime that schedules work across GPU clusters and own how quickly researchers iterate and how reliably long training runs complete.

You will collaborate with researchers to push capabilities, while improving performance, scheduling, and observability across experiments.

Qualifications

  • Depth in distributed systems and ML infrastructure for training and inference at scale.
  • Experience with Ray, Kubernetes, Slurm, or similar frameworks.
  • Proficient in Python and a systems language (Rust, C++, or Go).
  • Experience operating large-scale training or serving infrastructure, or schedulers for GPU workloads.
  • Strong debugging, profiling, and incident-diagnosis skills.
  • Ability to explain design and performance trade-offs to technical and non-technical audiences.

Responsibilities

  • Design, build, and operate distributed training and inference infrastructure for large language models, including Mixture-of-Experts models.
  • Scale training with parallelism strategies (data, tensor, pipeline, and expert), memory optimization, checkpointing, and failure recovery.
  • Build inference and rollout systems that support evaluation, experimentation, and RL without slowing down training.
  • Build execution environments and sandboxes for online RL and agent post-training.
  • Own continuous performance validation: dashboards showing rollout speed, training throughput, and learning behavior.

Skills

Distributed systems
Python
Systems programming
GPU clusters
Debugging & profiling
Communication of design choices

Tools

Ray
Kubernetes
Slurm
Megatron-LM
TorchTitan
DeepSpeed

Job description

Datadog Paris is seeking a Senior AI Research Engineer to join the Frontier Foundations team, building systems for large-scale LLM training, reinforcement learning, and inference. You will design and operate a distributed runtime that schedules work across GPU clusters and own how quickly researchers iterate and how reliably long training runs complete.

You will collaborate with researchers to push capabilities, while improving performance, scheduling, and observability across experiments.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI Research Engineer: LLM Training & Inference Infra
Senior AI Research Engineer: LLM Training & Inference Infra

Academic Key • Paris

On-site
EUR 120,000 - 180,000
RSUs
ESPP
Conference attendance
Senior AI Systems Engineer, Large-Scale Infra
Senior AI Systems Engineer, Large-Scale Infra

Datadog, Inc. • Paris

Hybrid
EUR 100,000 - 140,000
Stock options
Conference opportunities
Mentor program
+1
Senior AI GPU Kernel Engineer - LLM Training
Senior AI GPU Kernel Engineer - LLM Training

Academic Key • Paris

Hybrid
EUR 120,000 - 180,000
Hybrid work model
Senior AI Research Engineer - LLM Training & Inference Infrastructure
Senior AI Research Engineer - LLM Training & Inference Infrastructure

Academic Key • Paris

On-site
EUR 120,000 - 180,000
RSUs
ESPP
Conference attendance
Senior AI GPU Kernel Engineer — Hybrid, Paris
Senior AI GPU Kernel Engineer — Hybrid, Paris

Datadog, Inc. • Paris

Hybrid
EUR 90,000 - 130,000
Senior AI Research Engineer — GPU Kernels & LLM Training
Senior AI Research Engineer — GPU Kernels & LLM Training

ContentBuffer • Paris

Hybrid
EUR 90,000 - 120,000
Senior LLM Inference Engineer — GPU‑Driven Serving
Senior LLM Inference Engineer — GPU‑Driven Serving

Doctolib • Levallois-Perret

Hybrid
EUR 110,000 - 150,000
Health insurance
Lunch vouchers
Remote work flexibility
+2
Senior AI Research Engineer - GPU Systems & Kernels
Senior AI Research Engineer - GPU Systems & Kernels

Academic Key • Paris

Hybrid
EUR 120,000 - 180,000
Hybrid work model
Senior AI Research Engineer - LLM Training & Inference Infrastructure
Senior AI Research Engineer - LLM Training & Inference Infrastructure

Datadog, Inc. • Paris

On-site
EUR 120,000 - 180,000
RSUs and ESPP
Conference attendance
Mentor program
+1
Senior AI Research Engineer - LLM Training & Inference Infrastructure
Senior AI Research Engineer - LLM Training & Inference Infrastructure

Datadog, Inc. • Paris

Hybrid
EUR 100,000 - 140,000
Stock options
Conference opportunities
Mentor program
+1