Senior Inference & RL Systems Engineer (Scalable ML Infra)

Magic AI, Inc; San Francisco (CA); On-site; USD 300,000 - 550,000; Full time

Benefits offered by this job

Equity compensation

401(k) matching

Health, dental and vision insurance

Unlimited PTO

Visa sponsorship

Relocation stipend

Small, focused team

Job summary

Magic AI, Inc. is seeking a Member of Technical Staff to design and operate distributed systems for serving models in production and driving large-scale post-training workflows.

You will work where model execution meets distributed infrastructure, influencing latency, throughput, and reliability of RL and training loops. You will own the infrastructure enabling fast inference and scalable RL iteration, balancing KV-cache strategies, batching, and long-context workloads while collaborating with

Qualifications

• Strong software engineering and distributed systems fundamentals.
• Experience building or operating large-scale inference or training systems.
• Deep understanding of GPU execution constraints and memory trade-offs.
• Experience debugging performance issues in production ML systems.
• Ability to reason about system-level trade-offs between latency, throughput and cost.

Responsibilities

• Design and scale high-performance inference serving systems.
• Optimize KV-cache management, batching strategies, and scheduling.
• Improve throughput and latency for long-context workloads.
• Build and maintain distributed RL and post-training infrastructure.
• Improve reliability of rollout, evaluation, and reward pipelines.
• Automate fault detection and recovery for serving and RL systems.
• Profile and eliminate performance bottlenecks across GPU, networking and storage layers.
• Collaborate with Kernels and Research to align execution systems with model architecture.

Skills

Distributed systems

Production ML systems

GPU memory constraints

Performance debugging

Latency vs throughput

Ownership of production infra

Job description

Magic AI, Inc. is seeking a Member of Technical Staff to design and operate distributed systems for serving models in production and driving large-scale post-training workflows.

You will work where model execution meets distributed infrastructure, influencing latency, throughput, and reliability of RL and training loops. You will own the infrastructure enabling fast inference and scalable RL iteration, balancing KV-cache strategies, batching, and long-context workloads while collaborating with