RL Infrastructure Scientist — Scaled GPU Systems

AItoolnavio; Union (NJ); On-site; USD 180,000 - 280,000; Full time

Job summary

Luma is building reinforcement learning systems at frontier scale, coupling policy optimization with large fleets of inference workers and agentic environments. We seek engineers who can own the full RL post-training loop across thousands of GPUs, from training to rollout to evaluation.

You will design and optimize the distributed stack, improve throughput, and ensure stability in production runs, working with researchers to turn ideas into scalable systems.

Qualifications

• Hands-on experience post-training LLMs with RL (PPO/GRPO-family, RLHF, RLVR) at meaningful scale.
• Extensive distributed PyTorch training and parallelism (FSDP, Tensor/Pipeline/Expert Parallel) for foundation models.
• Experience building RL environments, reward functions, verifiers, or evaluation harnesses for LLM agents, including sandboxed execution and multi-turn tool use.
• Deep familiarity with RL post-training frameworks (veRL, OpenRLHF, TRL, Ray orchestration) and rollout inference engines (vLLM, SGLang).
• Strong understanding of GPU clusters, networking, and communication libraries (NCCL, MPI) under mixed training and inference workloads.

Responsibilities

• Design, build, and scale distributed RL post-training systems, orchestrating trainer, rollout, environment, and reward workloads across thousands of GPUs.
• Build high-throughput rollout generation, integrating inference engines (vLLM, SGLang), weight synchronization, and asynchronous/off-policy schemes.
• Design RL environments for agentic, multi-step tasks — sandboxed code execution, tool use, computer use, multimodal interaction — reproducible and scalable to millions of episodes.
• Build reward infrastructure: verifiable/programmatic rewards, reward-model serving, LLM-as-judge pipelines, and defenses against reward hacking.
• Develop the evaluation, monitoring, and debugging tooling that keeps large RL runs stable.
• Advance training efficiency and stability, and turn new post-training ideas into production runs with researchers.

Skills

RL post-training

Distributed PyTorch

RLLM frameworks

Rollout engines

NCCL MPI networks

Tools

vLLM

SGLang

Ray

Kubernetes

Job description

Luma is building reinforcement learning systems at frontier scale, coupling policy optimization with large fleets of inference workers and agentic environments. We seek engineers who can own the full RL post-training loop across thousands of GPUs, from training to rollout to evaluation.

You will design and optimize the distributed stack, improve throughput, and ensure stability in production runs, working with researchers to turn ideas into scalable systems.