AI Training Performance Engineer for 100B+ Scale Models

Figure; San Jose (CA); Presencial; USD 200.000 - 400.000; Jornada completa

Descripción de la vacante

Figure, an AI robotics company headquartered in San Jose, is seeking an experienced AI Training Performance Engineer to optimize large-scale model training across massive GPU clusters. You will collaborate on accelerator choices, build performance tools, and push innovations in kernels and data pipelines to maximize throughput.

The role requires deep GPU knowledge, strong Python/C++ skills, and a track record of scaling training workloads.

Formación

• Bachelor's or Master's degree in Computer Science, Computer/Electrical Engineering, or a related field.
• 3+ years in AI performance engineering with large-scale projects.
• Deep understanding of GPU architecture and performance characteristics.
• Proficiency with profiling tools and translating traces into optimizations.
• Strong Python and CUDA/C++ skills; reading and modifying framework internals.
• Experience with NCCL and modern networking concepts (RDMA, NVLink, InfiniBand).

Responsabilidades

• Optimize training performance for 100B+ parameter models across 100k+ GPUs.
• Collaborate on accelerator choice, cluster topology, scheduling, and hardware procurement decisions.
• Write and optimize custom kernels (Triton/CUDA).
• Build tooling and dashboards for performance monitoring, regression detection, and root-cause analysis.
• Optimize data loading and preprocessing pipelines to avoid I/O bottlenecks.
• Improve checkpointing, fault tolerance, and elastic restart for large jobs.
• Co-design model architectures and training recipes for performance at scale.
• Extend kernel compilers to target custom/non-NVIDIA accelerators.
• Build agentic systems that generate, benchmark, and iterate on custom kernels.
• Evaluate emerging accelerators (AMD, TPU, SRAM-based ASICs) for fit and port benchmarks.
• Explore model/data parallelisms to determine optimal configuration per model size.

Descripción del empleo

Figure, an AI robotics company headquartered in San Jose, is seeking an experienced AI Training Performance Engineer to optimize large-scale model training across massive GPU clusters. You will collaborate on accelerator choices, build performance tools, and push innovations in kernels and data pipelines to maximize throughput.

The role requires deep GPU knowledge, strong Python/C++ skills, and a track record of scaling training workloads.