Senior AI Training Performance Engineer (GPU & Scale)

figure.ai; San Jose (CA); Hybrid; USD 200,000 - 400,000; Full time

Job summary

Figure is seeking an experienced AI Training Performance Engineer to advance distributed training for massive models. The role focuses on optimizing GPU kernels, accelerator selection, and co-designing models to maximize hardware utilization.

You will develop kernels, build performance dashboards, and collaborate across teams to enable scalable, reliable training at scale. Applicants should have 3+ years in AI performance engineering, deep GPU knowledge, and strong Python and CUDA/C++ skills,

Qualifications

• 3+ years in AI performance engineering with large-scale projects.
• Deep understanding of GPU architecture and performance metrics.
• Strong Python and CUDA/C++ skills; capable of reading framework internals.
• Experience with profiling and translating traces into optimizations.
• Familiarity with high-performance interconnects (NCCL, RDMA, NVLink).

Responsibilities

• Optimize training performance for 100B+ parameter models across 100k+ GPUs.
• Collaborate on accelerator choice, cluster topology, scheduling, and hardware procurement.
• Write and optimize custom kernels (Triton/CUDA).
• Build tooling and dashboards for performance monitoring and regression detection.
• Improve data loading, checkpointing, fault tolerance, and elastic restart.
• Co-design model architectures and training recipes for scale.

Skills

GPU architecture

Profiling tools

Python

CUDA/C++

NCCL

Distributed training

Performance engineering

Education

Bachelor's or Master's degree in CS/EE

Tools

Nsight Systems/Compute

PyTorch Profiler

HTA

Job description

Figure is seeking an experienced AI Training Performance Engineer to advance distributed training for massive models. The role focuses on optimizing GPU kernels, accelerator selection, and co-designing models to maximize hardware utilization.

You will develop kernels, build performance dashboards, and collaborate across teams to enable scalable, reliable training at scale. Applicants should have 3+ years in AI performance engineering, deep GPU knowledge, and strong Python and CUDA/C++ skills,