Distributed Training Engineer for Multimodal Models

Luma; Redwood City (CA); On-site; USD 210,000 - 260,000; Full time

Job summary

Luma is seeking an engineer to design and optimize distributed training systems for multimodal models across thousands of GPUs. You will tackle advanced parallelism, training stability, and utilization at scale.

You will work with PyTorch, CUDA, NCCL, and Tensor Parallel, building monitoring and tooling for large-scale runs. This role suits someone who has shipped foundation-model training at scale and can improve stability and efficiency on massive clusters.

Qualifications

• Experience in extensive distributed PyTorch training and parallelisms for foundation-models.
• Deep understanding of GPU clusters, networking, and storage systems.
• Familiarity with communication libraries (NCCL, MPI) and distributed-system optimization.

Responsibilities

• Design, implement, and optimize distributed training systems for models across thousands of GPUs.
• Research and implement advanced parallelization (FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel).
• Build monitoring, visualization, and debugging tools for large-scale training runs.
• Optimize training stability, convergence, and resource utilization across massive clusters.
• Immerse & diagnose training stacks, land a parallelization or stability improvement, and scale tooling for reliability.

Skills

Distributed PyTorch training

GPU clusters & networking

Distributed-system optimization

Tools

NCCL

MPI

Job description

Luma is seeking an engineer to design and optimize distributed training systems for multimodal models across thousands of GPUs. You will tackle advanced parallelism, training stability, and utilization at scale.

You will work with PyTorch, CUDA, NCCL, and Tensor Parallel, building monitoring and tooling for large-scale runs. This role suits someone who has shipped foundation-model training at scale and can improve stability and efficiency on massive clusters.