Member of Technical Staff - Mid-Training Infra

Reflection; New York (NY); On-site; USD 150,000 - 200,000; Full time

Benefits offered by this job

Comprehensive medical, dental, vision, life, and disability insurance

Fully paid parental leave

Daily meals provided

Relocation support

Paid time off

Job summary

Reflection seeks a skilled professional in New York to design and operate large-scale GPU infrastructure for model inference and reinforcement learning. The role demands several years of experience in deploying GPU systems, optimizing model performance, and working with frameworks like SGLang and Megatron. The position offers competitive compensation, comprehensive benefits, and a supportive work environment with daily meals and team activities.

Qualifications

• Several years of hands-on experience building and running production infrastructure.
• Strong understanding of GPU performance characteristics and optimization techniques.
• Experience deploying large-scale GPU systems for inference or model serving.

Responsibilities

• Design and operate large-scale GPU infrastructure for model inference.
• Develop systems for synthetic data generation and reinforcement learning pipelines.
• Optimize GPU utilization for large language model inference.

Skills

GPU system deployment

Model serving

Optimizing throughput

Distributed reinforcement learning

Performance optimization techniques

Tools

SGLang

Megatron

Job description

About The Role

• Design, build, and operate large-scale GPU infrastructure for high-throughput model inference and mid-training workloads.
• Develop systems that power synthetic data generation and reinforcement learning pipelines at scale.
• Build high-performance inference platforms capable of serving and evaluating models across thousands of GPUs.
• Optimize throughput, latency, and GPU utilization for large language model inference and rollout workloads.
• Build infrastructure that supports reinforcement learning pipelines, including large-scale rollout generation, evaluation, and policy improvement loops.
• Work closely with research teams to support distributed RL workloads and large-scale model evaluation infrastructure.
• Improve performance of model execution through kernel-level optimization, model parallelism strategies, and GPU runtime improvements.
• Develop distributed systems that enable large-scale synthetic data generation and RL-driven training workflows.
• Diagnose and resolve performance bottlenecks across inference runtimes, GPU kernels, networking, and distributed compute systems.

Ideal Experience

• Experience deploying and operating large-scale GPU systems for inference or model serving.
• Several years of hands-on experience building and running production infrastructure.
• Strong understanding of GPU performance characteristics and optimization techniques.
• Experience working with modern inference frameworks such as SGLang, Megatron, or similar high-performance LLM runtimes.
• Familiarity with distributed reinforcement learning infrastructure or rollout generation systems.
• Experience optimizing throughput for large-scale model execution workloads.
• Experience working with GPU kernels or low-level performance optimization.
• Familiarity with infrastructure used for synthetic data pipelines or RL training workflows.
• Experience debugging performance issues across GPU, networking, and distributed execution layers.

What We Offer

• Top-tier compensation: Salary and equity structured to recognize and retain the best talent globally.
• Health & wellness: Comprehensive medical, dental, vision, life, and disability insurance.
• Life & family: Fully paid parental leave for all new parents, including adoptive and surrogate journeys. Financial support for family planning.
• Benefits & balance: paid time off when you need it, relocation support, and more perks that optimize your time.
• Opportunities to connect with teammates: lunch and dinner are provided daily. We have regular off-sites and team celebrations.