Senior AI GPU Kernel Engineer - High-Performance LLM Training (Hybrid)

Datadog, Inc.

Paris (KY)

Hybrid

USD 150,000 - 230,000

Full time

8 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Datadog, Inc. is seeking a Research Engineer on the Frontier Foundations team to accelerate large-scale model training and inference by writing and tuning GPU kernels and removing bottlenecks on modern GPUs.

You will collaborate with Research Scientists to build high-performance infrastructure to train and serve large language models, explore low-precision training and fused kernels, and optimize memory and communication across distributed setups.

Qualifications

  • Deep experience in GPU kernel development and optimization (CUDA or similar).
  • Solid understanding of memory hierarchies and interconnects for scalable training.
  • Proficient in system languages (C++, Rust, or CUDA C) and Python.
  • Strong debugging and profiling skills with tools like Nsight.

Responsibilities

  • Write, tune, and debug GPU kernels for LLM training and inference.
  • Optimize for latest NVIDIA GPUs, including tensor cores and FP formats.
  • Reduce communication overhead in distributed training and overlap with compute.
  • Profile end-to-end workloads and identify bottlenecks.
  • Develop benchmarks, profiling tools, and regression tests.
  • Collaborate with Research Scientists to implement new model architectures.
  • Explore low-precision training, speculative decoding, and fused/persistent kernels.

Skills

GPU kernel development
CUDA / Triton optimization
Low-latency distributed training
C++, Rust, Python
Profiling with Nsight
Cross-team communication

Tools

NVIDIA GPU architectures
NCCL / NVSHMEM

Job description

Datadog, Inc. is seeking a Research Engineer on the Frontier Foundations team to accelerate large-scale model training and inference by writing and tuning GPU kernels and removing bottlenecks on modern GPUs.

You will collaborate with Research Scientists to build high-performance infrastructure to train and serve large language models, explore low-precision training and fused kernels, and optimize memory and communication across distributed setups.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

GPU Kernel Architect for Fast LLM Training & Inference
GPU Kernel Architect for Fast LLM Training & Inference

Datadog, Inc. • Paris (TX)

On-site
USD 170,000 - 210,000
Senior AI Research Engineer - GPU Systems & Kernels
Senior AI Research Engineer - GPU Systems & Kernels

Datadog, Inc. • Paris (TX)

On-site
USD 170,000 - 210,000
Senior AI Infra Engineer, Large-Scale ML Systems
Senior AI Infra Engineer, Large-Scale ML Systems

Datadog, Inc. • Paris (TX)

Hybrid
USD 180,000 - 240,000
Stock options
ESPP
Global collaboration across offices
+1
Senior AI Research Engineer - GPU Systems & Kernels
Senior AI Research Engineer - GPU Systems & Kernels

Datadog, Inc. • Paris (KY)

Hybrid
USD 150,000 - 230,000
Senior AI Infra Engineer: Large-Scale LLM Training
Senior AI Infra Engineer: Large-Scale LLM Training

Datadog, Inc. • Paris (KY)

Hybrid
USD 180,000 - 260,000
RSUs/ESPP
Conferences & meetups
Mentor program
+2
GPU Kernel Engineer for Fast AI Training & Inference
GPU Kernel Engineer for Fast AI Training & Inference

River AI • Palo Alto (CA)

On-site
USD 200,000 - 420,000
Equity
Visa sponsorship
Relocation assistance
+1
Remote GPU Performance Engineer for Large-Scale Models
Remote GPU Performance Engineer for Large-Scale Models

United States Digital Space LLC • United States

Remote
USD 140,000 - 210,000
Five weeks paid leave
Comprehensive healthcare (vision +</p>
GPU Kernel Engineer — Accelerate AI Training & Inference
GPU Kernel Engineer — Accelerate AI Training & Inference

River AI Inc. • Palo Alto (CA)

On-site
USD 200,000 - 420,000
Health insurance
Dental insurance
Vision insurance
+3
Frontier AI Kernel & Performance Engineer
Frontier AI Kernel & Performance Engineer

Zyphra Technologies Inc. • San Francisco (CA)

On-site
USD 180,000 - 260,000
Medical, dental, vision, and FSA plans
401(k) plan
Relocation and immigration support
+3
Senior AI Runtime Systems Engineer - GPU Training
Senior AI Runtime Systems Engineer - GPU Training

The Briefed • Mountain View (CA)

On-site
USD 190,000 - 240,000
Equity grants
Bonus eligibility
Healthcare coverage
+2