Staff Engineer - Large-Scale GPU Inference & RL Infra

reflectionai; San Francisco, New York (CA, NY); On-site; USD 180,000 - 320,000; Full time

Benefits offered by this job

Top-tier compensation

Stock options

Health & wellness

Meals provided

Parental leave

Unlimited vacation

Visa sponsorship

Team events

Job summary

Reflection AI is building open, scalable AI infrastructure to enable high-throughput model inference and mid-training workloads. You will design and operate systems that support synthetic data generation, distributed RL, and evaluation across thousands of GPUs.

You will optimize throughput, latency, and GPU utilization while collaborating with research teams to advance large-scale model evaluation and deployment.

Qualifications

• Experience deploying and operating large-scale GPU systems for inference or model serving.
• Several years of hands-on experience building and running production infrastructure.
• Strong understanding of GPU performance characteristics and optimization techniques.
• Experience working with modern inference frameworks such as SGLang, Megatron, or similar high-performance LLM runtimes.
• Familiarity with distributed reinforcement learning infrastructure or rollout generation systems.
• Experience optimizing throughput for large-scale model execution workloads.
• Experience working with GPU kernels or low-level performance optimization.
• Familiarity with infrastructure used for synthetic data pipelines or RL training workflows.
• Experience debugging performance issues across GPU, networking, and distributed execution layers.

Responsibilities

• Design, build, and operate large-scale GPU infrastructure for high-throughput model inference and mid-training workloads.
• Develop systems that power synthetic data generation and reinforcement learning pipelines at scale.
• Build high-performance inference platforms capable of serving and evaluating models across thousands of GPUs.
• Diagnose and resolve performance bottlenecks across inference runtimes, GPU kernels, networking, and distributed compute systems.

Skills

GPU infrastructure

Model serving

RL pipelines

Distributed systems

Kernel optimization

Tools

SGLang

Megatron

Job description

Reflection AI is building open, scalable AI infrastructure to enable high-throughput model inference and mid-training workloads. You will design and operate systems that support synthetic data generation, distributed RL, and evaluation across thousands of GPUs.

You will optimize throughput, latency, and GPU utilization while collaborating with research teams to advance large-scale model evaluation and deployment.