Senior ML Performance Architect - GPU & HPC Leader

Proxima; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Proxima is seeking a Principal ML Performance Engineer to optimize GPU workloads for large-scale models and diffusion-based pipelines. You will profile, tune, and accelerate training and inference on multi-node clusters, with emphasis on memory and kernel efficiency.

Your work drives cost and speed improvements across 32–64 GPUs. You’ll implement custom kernels, use advanced compilers, and collaborate with researchers to push the limits of PyTorch, Triton, and CUDA.

Qualifications

• Minimum of 6+ years experience in ML systems, HPC, or performance engineering.
• Ability to set technical direction beyond coding: selecting infrastructure, influencing research teams, and mentoring engineers.
• Deep knowledge of PyTorch internals with hands-on profiling and fixing real bottlenecks.
• Experience with CUDA and Triton, reading Nsight output, and strong understanding of memory bandwidth and occupancy.
• Experience with distributed training at multi-node scale.
• Strong proficiency in Python and C++.
• Able to name a model they made materially faster and quantify the improvement.

Responsibilities

• Profile and optimize training and inference for structural and generative models, including transformers, diffusion, and geometric deep learning.
• Write and tune custom kernels (CUDA, Triton) and use compilers (torch.compile, TensorRT, XLA) when beneficial
• Scale distributed training across 32-64 nodes, employing FSDP, DeepSpeed, tensor and pipeline parallelism, and mixed precision
• Reduce inference cost by optimizing memory scaling for large complexes, improving diffusion sampling efficiency, batching ragged inputs, and maximizing throughput across up to 1000 GPUs
• Manage GPU cluster efficiency on GCP, focusing on scheduling, utilization, spot strategy, and cost reporting
• Develop benchmarks and profiling tools for the research team

Skills

PyTorch internals

CUDA

Triton

Python

C++

Distributed training

Profiling bottlenecks

GPU memory optimization

Nsight profiling

Education

BS/MS/PhD in CS, EE, or related field

Tools

Nsight

TensorRT

XLA

Torch.compile

Job description

Proxima is seeking a Principal ML Performance Engineer to optimize GPU workloads for large-scale models and diffusion-based pipelines. You will profile, tune, and accelerate training and inference on multi-node clusters, with emphasis on memory and kernel efficiency.

Your work drives cost and speed improvements across 32–64 GPUs. You’ll implement custom kernels, use advanced compilers, and collaborate with researchers to push the limits of PyTorch, Triton, and CUDA.