Distributed Training Engineer for Large Transformer Models

Axiōma Search; Greater London; Hybrid; GBP 90,000 - 150,000; Full time

Job summary

Axiōma Search is building state-of-the-art foundation-model training infrastructure in London. This early hire will own distributed training end-to-end, scaling from tens to thousands of GPUs, and shape long-term infra decisions. You will stay hands-on, writing code while guiding the stack’s evolution.

You’ll work on-site five days a week in London, contributing to high-performance GPU clusters and cloud-based training pipelines as the product develops.

Qualifications

• Hands-on experience training large transformer models across multi-node GPU clusters.
• Deep distributed-training experience with PyTorch, NCCL, DeepSpeed, Megatron-LM/FSDP.
• Cloud-based GPU workloads, ideally AWS.
• Track record improving utilisation, networking, storage, checkpointing and reliability.
• Able to lead this area independently while contributing code.

Responsibilities

• Own distributed training infrastructure and scale it across large GPU clusters.
• Push training from tens of GPUs to hundreds, and thousands.
• Improve GPU utilization, throughput and end-to-end training efficiency.
• Optimise networking, storage, checkpointing and data pipelines for large jobs.
• Profile and optimise across the full training stack.
• Make long-term infrastructure and tooling decisions.
• Potentially support cloud inference and deployment as the product develops.

Skills

Distributed training

PyTorch

NCCL

DeepSpeed

Megatron-LM

AWS

GPU utilization

On-site London

Job description

Axiōma Search is building state-of-the-art foundation-model training infrastructure in London. This early hire will own distributed training end-to-end, scaling from tens to thousands of GPUs, and shape long-term infra decisions. You will stay hands-on, writing code while guiding the stack’s evolution.

You’ll work on-site five days a week in London, contributing to high-performance GPU clusters and cloud-based training pipelines as the product develops.