Staff ML Systems Engineer - Distributed Training & Inference

RadixArk; Palo Alto (CA); On-site; USD 200,000 - 400,000; Full time

Benefits offered by this job

401(k) plan

Unlimited PTO

Visa sponsorship

Job summary

RadixArk is seeking a Member of Technical Staff, Training, to design, build, and operate the distributed systems behind large-scale model post-training. You will work across training, inference, and orchestration on multi-GPU clusters to ensure performance, correctness, and reliability.

The role blends modeling, infrastructure, and low-level systems work, with opportunities to contribute to Miles, SGLang, and other open-source projects.

Qualifications

• 3+ years of experience building or operating distributed ML systems.
• Hands-on experience with post-training backends or inference systems for large language models (e.g., Megatron-LM, FSDP, SGLang, TensorRT-LLM, vLLM).
• Experience in at least two of the following areas: performance/scalability of multi-GPU, multi-node workloads; numerical correctness or low precision; stability or fault tolerance; post-training recipes and orchestration for large training runs; multimodal training or inference; agent infrastructure and sandboxed eval systems; open-source projects.

Responsibilities

• Design, build, and operate distributed training, rollout, and orchestration systems for large-scale LLM and multimodal post-training across multi-GPU, multi-node environments.
• Profile and optimize performance across the full-stack — model implementation, parallelism strategies, communication libraries, and GPU kernels.
• Investigate numerical correctness and low-precision issues in distributed training and inference, including train–inference consistency for reinforcement learning.
• Improve the reliability of long-running workloads through checkpointing, fault recovery, observability, and operational tooling.
• Build supporting infrastructure for reinforcement learning and agentic post-training, including asynchronous rollout, trajectory collection, sandboxed execution, evaluation harnesses, and data pipelines.
• Contribute to open-source training and inference systems, including Miles and SGLang, and partner with researchers to turn experimental requirements into production systems.

Skills

Distributed ML systems

Multi-GPU training

High-performance inference

Tools

Miles

SGLang

Megatron-LM

TensorRT-LLM

vLLM

NCCL

RDMA

NVLink/NVSwitch

CUDA

Triton

CUTLASS

Job description

RadixArk is seeking a Member of Technical Staff, Training, to design, build, and operate the distributed systems behind large-scale model post-training. You will work across training, inference, and orchestration on multi-GPU clusters to ensure performance, correctness, and reliability.

The role blends modeling, infrastructure, and low-level systems work, with opportunities to contribute to Miles, SGLang, and other open-source projects.