Large-Scale GPU Training Engineer & Scientist

Luma; New York (NY); Sur place; USD 195 000 - 395 000; Plein temps

Résumé du poste

Luma seeks an experienced engineer to design and optimize distributed training systems across thousands of GPUs. You will advance parallel strategies like FSDP, tensor, and pipeline parallelism while building tools to monitor and debug massive training runs.

In this role you’ll ensure training stability, efficient resource use, and scalability on large GPU clusters, contributing to research at scale. Strong Linux, networking, and cloud fundamentals are valued.

Qualifications

• Extensive distributed PyTorch training experience in foundation-models.
• Deep understanding of GPU clusters, networking, and storage systems.
• Familiarity with communication libraries (NCCL, MPI) and distributed-system optimization.

Responsabilités

• Design, implement, and optimize distributed training systems for models across thousands of GPUs.
• Research and implement advanced parallelization (FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel).
• Build monitoring, visualization, and debugging tools for large-scale training runs.
• Optimize training stability, convergence, and resource utilization across massive clusters.

Connaissances

Distributed PyTorch training

GPU clusters

NCCL/MPI

Outils

NCCL

MPI

Description du poste

Luma seeks an experienced engineer to design and optimize distributed training systems across thousands of GPUs. You will advance parallel strategies like FSDP, tensor, and pipeline parallelism while building tools to monitor and debug massive training runs.

In this role you’ll ensure training stability, efficient resource use, and scalability on large GPU clusters, contributing to research at scale. Strong Linux, networking, and cloud fundamentals are valued.