Senior Distributed ML Systems Engineer

Luma; Greater London; On-site; GBP 147,000 - 299,000; Full time

Job summary

Luma is building distributed training systems for its large-scale multimodal models. This role focuses on PyTorch, CUDA, and advanced parallelism across thousands of GPUs, delivering reliable, scalable infrastructure so researchers can innovate.

You will design and optimize training systems, implement FSDP, Tensor Parallel, Pipeline Parallel, and Expert Parallel, and build monitoring and debugging tools to improve stability and utilization.

Qualifications

• Extensive distributed PyTorch training and parallelism in foundation-model training.
• Deep understanding of GPU clusters, networking, and storage systems.
• Familiarity with communication libraries (NCCL, MPI) and distributed-system optimization.
• Experience managing training runs across 100+ GPUs.

Responsibilities

• Design, implement, and optimize distributed training systems for models across thousands of GPUs.
• Research and implement advanced parallelization (FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel).
• Build monitoring, visualization, and debugging tools for large-scale training runs.
• Optimize training stability, convergence, and resource utilization across massive clusters.

Skills

Distributed PyTorch training

GPU clusters

Networking and storage

NCCL/MPI

Tools

PyTorch

NCCL

MPI

Job description

Luma is building distributed training systems for its large-scale multimodal models. This role focuses on PyTorch, CUDA, and advanced parallelism across thousands of GPUs, delivering reliable, scalable infrastructure so researchers can innovate.

You will design and optimize training systems, implement FSDP, Tensor Parallel, Pipeline Parallel, and Expert Parallel, and build monitoring and debugging tools to improve stability and utilization.