Staff Engineer, Inference & RL Systems — Scale ML

magic.dev; San Francisco (CA); On-site; USD 275,000 - 550,000; Full time

Benefits offered by this job

Equity

401(k) matching

Health insurance

Unlimited PTO

Visa sponsorship

Relocation stipend

Small, focused team

Job summary

Magic is building safe AGI and operates distributed serving and RL infrastructure. As a Research Engineer on the Inference &RL Systems team, you will design and operate the systems that serve our models in production and power large-scale post-training workflows.

You will own the infrastructure that makes production inference fast and reliable, addressing KV-cache scaling, long-context workloads, and throughput under real-world workloads while collaborating with Kernels and Research.

Qualifications

• Strong software engineering and distributed systems fundamentals.
• Experience building or operating large-scale inference or training systems.
• Deep understanding of GPU execution constraints and memory trade-offs.
• Experience debugging performance issues in production ML systems.
• Ability to reason about system-level trade-offs between latency, throughput, and cost.
• Track record of owning critical production infrastructure.

Responsibilities

• Design and scale high-performance inference serving systems
• Optimize KV-cache management, batching strategies, and scheduling
• Improve throughput and latency for long-context workloads
• Build and maintain distributed RL and post-training infrastructure
• Improve reliability of rollout, evaluation, and reward pipelines
• Automate fault detection and recovery for serving and RL systems
• Profile and eliminate performance bottlenecks across GPU, networking, and storage layers
• Collaborate with Kernels and Research to align execution systems with model architecture

Skills

Distributed systems

Inference systems

GPU memory constraints

Performance debugging

System trade-offs

Production infra ownership

Job description

Magic is building safe AGI and operates distributed serving and RL infrastructure. As a Research Engineer on the Inference &RL Systems team, you will design and operate the systems that serve our models in production and power large-scale post-training workflows.

You will own the infrastructure that makes production inference fast and reliable, addressing KV-cache scaling, long-context workloads, and throughput under real-world workloads while collaborating with Kernels and Research.