Staff Software Engineer, GPU Infra & ML Systems

Reflection AI Ltd; New York (NY); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Top-tier compensation

Stock options

Health & wellness

Meals provided

Parental leave

Unlimited PTO (US)

Visa sponsorship

Job summary

Reflection AI Ltd. is seeking an experienced engineer to design, build, and operate large-scale GPU infrastructure for high-throughput model inference and mid-training workloads.

You will develop systems powering synthetic data generation and reinforcement learning pipelines at scale, and build platforms capable of serving thousands of GPUs. Ideal candidates have hands-on experience with production infrastructure, strong GPU optimization expertise, and familiarity with advanced inference

Qualifications

• Experience deploying and operating large-scale GPU systems for inference or model serving.
• Several years of hands-on experience building and running production infrastructure.
• Strong understanding of GPU performance characteristics and optimization techniques.
• Experience with modern inference frameworks such as SGLang or Megatron.
• Familiarity with distributed reinforcement learning infrastructure or rollout generation systems.
• Experience optimizing throughput for large-scale model execution workloads.
• Experience with GPU kernels or low-level performance optimization.
• Familiarity with infrastructure used for synthetic data pipelines or RL training workflows.
• Experience debugging performance issues across GPU, networking, and distributed execution layers.

Responsibilities

• Design, build, and operate large-scale GPU infrastructure for high-throughput model inference and mid-training workloads.
• Develop systems that power synthetic data generation and reinforcement learning pipelines at scale.
• Build high-performance inference platforms capable of serving and evaluating models across thousands of GPUs.
• Optimize throughput, latency, and GPU utilization for large language model inference and rollout workloads.
• Build infrastructure that supports reinforcement learning pipelines, including large-scale rollout generation, evaluation, and policy improvement loops.
• Work closely with research teams to support distributed RL workloads and large-scale model evaluation infrastructure.
• Improve performance of model execution through kernel-level optimization, model parallelism strategies, and GPU runtime improvements.
• Develop distributed systems that enable large-scale synthetic data generation and RL-driven training workflows.
• Diagnose and resolve performance bottlenecks across inference runtimes, GPU kernels, networking, and distributed compute systems.

Skills

GPU deployment

Production infrastructure

Distributed RL

Performance optimization

Kernel-level optimization

Tools

Megatron

SGLang

LLM runtimes

Job description

Reflection AI Ltd. is seeking an experienced engineer to design, build, and operate large-scale GPU infrastructure for high-throughput model inference and mid-training workloads.

You will develop systems powering synthetic data generation and reinforcement learning pipelines at scale, and build platforms capable of serving thousands of GPUs. Ideal candidates have hands-on experience with production infrastructure, strong GPU optimization expertise, and familiarity with advanced inference