ML Systems Engineer — GPU & Distributed Inference

True; Menlo Park (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

The company is seeking an experienced ML systems engineer to optimize training and inference workloads for image, video, diffusion, and world-model tasks at their Menlo Park, CA site. You will profile bottlenecks, implement low-level GPU optimizations with CUDA and Triton, and help design distributed engines for diffusion models.

This role requires 2+ years in ML systems, strong CS fundamentals, and hands-on experience with PyTorch, Nsight, NCCL, and RDMA.

Qualifications

• 2+ years in deep learning training or inference systems with ML systems or GPU performance experience.
• Experience authoring core features in an inference or training framework (e.g., vLLM, SGLang, TensorRT-LLM, Megatron).
• Hands-on GPU kernel development for NVIDIA hardware using CUDA, CUTLASS, Triton, or PTX.
• Strong machine learning and CS fundamentals, usually with a CS-related degree.
• Experience with PyTorch, Nsight, NCCL, and RDMA.
• Experience optimizing inference or training performance for diffusion, video, image, or multimodal workloads.

Responsibilities

• Optimize training and inference performance for image, video, diffusion, and world-model workloads.
• Profile bottlenecks across kernels, memory, systems, and clusters using Nsight and related tools.
• Implement GPU optimizations using CUDA and Triton.
• Design and improve distributed training and inference engines for diffusion models.
• Improve communication performance across GPUs and nodes using NCCL, RDMA, InfiniBand, and RoCE.
• Build benchmarks and regression tests to ensure performance improvements hold in production.
• Contribute to hardware-aware co-design across kernels, runtimes, and models.

Skills

ML systems

GPU performance

CUDA

Triton

PyTorch

Distributed systems

Education

Bachelor's degree in CS or related field

Tools

Nsight

NCCL

RDMA

InfiniBand

RoCE

CUDA

CUTLASS

Triton

Job description

The company is seeking an experienced ML systems engineer to optimize training and inference workloads for image, video, diffusion, and world-model tasks at their Menlo Park, CA site. You will profile bottlenecks, implement low-level GPU optimizations with CUDA and Triton, and help design distributed engines for diffusion models.

This role requires 2+ years in ML systems, strong CS fundamentals, and hands-on experience with PyTorch, Nsight, NCCL, and RDMA.