Senior Distributed ML Engineer - Large-Scale GPU Training

Luma; United States; Sur place; USD 195 000 - 395 000; Plein temps

Résumé du poste

Luma is building distributed systems to train large-scale multimodal models across thousands of GPUs, delivering reliable, efficient infrastructure for researchers to innovate on top of. This role focuses on PyTorch, CUDA, and advanced parallelism to scale foundation-model training.

You will design and optimize training stacks, implement parallelization techniques (FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel), and build monitoring and tooling to keep runs stable and productive at

Qualifications

• Extensive distributed PyTorch training and parallelisms in foundation-model training.
• Deep understanding of GPU clusters, networking, and storage systems.
• Familiarity with communication libraries (NCCL, MPI) and distributed-system optimization.

Responsabilités

• Design, implement, and optimize efficient distributed training systems for models across thousands of GPUs.
• Research and implement advanced parallelization (FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel).
• Build monitoring, visualization, and debugging tools for large-scale training runs.
• Optimize training stability, convergence, and resource utilization across massive clusters.
• Immerse & diagnose current training stack and stability issues; land improvements.

Connaissances

Distributed PyTorch

GPU clusters

NCCL/MPI

Linux

Outils

Containerization

Description du poste

Luma is building distributed systems to train large-scale multimodal models across thousands of GPUs, delivering reliable, efficient infrastructure for researchers to innovate on top of. This role focuses on PyTorch, CUDA, and advanced parallelism to scale foundation-model training.

You will design and optimize training stacks, implement parallelization techniques (FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel), and build monitoring and tooling to keep runs stable and productive at