Senior Multi-GPU Distributed Training Engineer

Luma; Seattle (WA); On-site; USD 195,000 - 395,000; Full time

Job summary

Luma is seeking an engineer to build and optimize distributed training systems for foundation-model scale across thousands of GPUs in Seattle. You’ll own performance, stability, and efficiency of multi-node runs, collaborating with researchers to push the frontiers of multimodal AI.

You bring deep experience with distributed PyTorch, FSDP, Tensor Parallel, and large GPU clusters, plus strong Linux scripting, networking, and tooling to monitor, diagnose, and scale training workloads.

Qualifications

• Extensive distributed PyTorch training across large GPU clusters.
• Experience with multi-node training, stability, and efficiency at scale.
• Familiarity with FSDP, Tensor Parallel, Pipeline Parallel, and Expert Parallel is a plus.

Responsibilities

• Design and optimize distributed training systems for models across thousands of GPUs.
• Build monitoring, visualization, and debugging tooling for large-scale runs.
• Collaborate with researchers to improve training stability and utilization.

Skills

Distributed PyTorch

FSDP

Tensor Parallel

Pipeline Parallel

GPU Clusters

Networking

MPI/NCCL

Performance Tuning

Monitoring Tools

Tools

NVIDIA NCCL

MPI

Linux

Containers

Job description

Luma is seeking an engineer to build and optimize distributed training systems for foundation-model scale across thousands of GPUs in Seattle. You’ll own performance, stability, and efficiency of multi-node runs, collaborating with researchers to push the frontiers of multimodal AI.

You bring deep experience with distributed PyTorch, FSDP, Tensor Parallel, and large GPU clusters, plus strong Linux scripting, networking, and tooling to monitor, diagnose, and scale training workloads.