Distributed Training Engineer for Large-Scale GPU Clusters

Luma; Greater London; On-site; GBP 120,000 - 160,000; Full time

Job summary

Luma seeks an engineer to build and optimize distributed training systems for multimodal models across thousands of GPUs. You will tackle advanced parallelism, training stability, and utilization across massive clusters to empower researchers with reliable, scalable infrastructure.

This role focuses on real problems at scale, including FSDP, multi-node training, and efficient communication. You will develop monitoring tools and automation to keep large runs stable and productive.

Qualifications

• Extensive distributed PyTorch training experience on large models.
• Deep understanding of GPU clusters, networking, and storage.
• Familiarity with NCCL, MPI and distributed-system optimization.

Responsibilities

• Design, implement, and optimize distributed training systems for models across thousands of GPUs.
• Research and implement advanced parallelization (FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel).
• Build monitoring, visualization, and debugging tools for large-scale training runs.
• Optimize training stability, convergence, and resource utilization across massive clusters.

Skills

Distributed PyTorch training

GPU clusters

NCCL/MPI

Linux scripting

Tools

CUDA

Tensor Parallel

FSDP

Job description

Luma seeks an engineer to build and optimize distributed training systems for multimodal models across thousands of GPUs. You will tackle advanced parallelism, training stability, and utilization across massive clusters to empower researchers with reliable, scalable infrastructure.

This role focuses on real problems at scale, including FSDP, multi-node training, and efficient communication. You will develop monitoring tools and automation to keep large runs stable and productive.