RL Infrastructure Architect — Large-Scale Training Systems

Lumaai; Greater London; On-site; GBP 150,000 - 210,000; Full time

Job summary

Lumaai is hiring to build systems that power reinforcement learning at frontier scale. You will design and scale post-training RL stacks across thousands of GPUs, orchestrating trainer, rollout, environment, and reward workloads with high-throughput throughput and verifiable rewards.

You will develop RL environments, reward models, and evaluation harnesses for LLM agents, leveraging vLLM, SGLang, and Ray ecosystems. Expect deep collaboration with researchers to push stability and efficiency.

Qualifications

• Hands-on experience with post-training LLMs using RL (PPO/GRPO-family, RLHF, RLVR) at meaningful scale.
• Extensive distributed PyTorch training and parallelism (FSDP, Tensor/Pipeline/Expert) for foundation models.
• Experience building RL environments, reward functions, verifiers, or evaluation harnesses for LLM agents, including sandboxed execution and multi-turn tool use.
• Deep familiarity with RL post-training frameworks (veRL, OpenRLHF, TRL, Ray orchestration) and rollout inference engines (vLLM, SGLang).
• Strong understanding of GPU clusters, networking, and communication libraries (NCCL, MPI) under mixed training and inference workloads.

Responsibilities

• Design, build, and scale distributed RL post-training systems, orchestrating trainer, rollout, environment, and reward workloads across thousands of GPUs.
• Build high-throughput rollout generation, integrating inference engines (vLLM, SGLang), weight synchronization, and asynchronous/off-policy schemes.
• Design RL environments for agentic, multi-step tasks — sandboxed code execution, tool use, computer use, multimodal interaction — reproducible and scalable to millions of episodes.
• Build reward infrastructure: verifiable/programmatic rewards, reward-model serving, LLM-as-judge pipelines, and defenses against reward hacking.
• Develop the evaluation, monitoring, and debugging tooling that keeps large RL runs stable.
• Advance training efficiency and stability, and turn new post-training ideas into production runs with researchers.

Skills

RL post-training LLMs with PPO/RLHF

Distributed PyTorch training

GPU cluster management

RL evaluation & verification tooling

Rollout inference engines

NCCL/MPI networking

Kubernetes/Ray orchestration

Tools

veRL

OpenRLHF

TRL

Ray

vLLM

SGLang

NCCL

Job description

Lumaai is hiring to build systems that power reinforcement learning at frontier scale. You will design and scale post-training RL stacks across thousands of GPUs, orchestrating trainer, rollout, environment, and reward workloads with high-throughput throughput and verifiable rewards.

You will develop RL environments, reward models, and evaluation harnesses for LLM agents, leveraging vLLM, SGLang, and Ray ecosystems. Expect deep collaboration with researchers to push stability and efficiency.