Staff RL Environments Engineer: Build Scalable AI Worlds

Scale AI; San Francisco (CA); On-site; USD 252,000 - 315,000; Full time

Benefits offered by this job

Health benefits

Dental benefits

Vision benefits

Retirement benefits

Learning stipend

Generous PTO

Commuter stipend

Job summary

Scale AI is seeking a Staff Software Engineer, RL Environments, to own the technical foundation for large-scale RL environments. You will design sandboxed execution, packaging/versioning, rollout orchestration, and verifier frameworks, while instrumenting real applications and building task suites to reveal capability gaps.

In this hands-on role, you will lead across teams, write the challenging code, and ensure trustworthy reward signals and throughput for millions of rollouts at scale.

Qualifications

• 8+ years of software engineering experience with strong fundamentals in distributed systems, system design, data structures, and algorithms.
• Strong Python skills and a track record of shipping production software; comfort in at least one other part of the stack (TypeScript/React, Go, Rust, or similar).
• Deep experience with containerization and sandboxed execution, including Docker, VMs, gVisor/Firecracker, Kubernetes, or equivalent.
• Experience building or operating high-throughput backend systems: orchestration, job scheduling, queuing, and large-scale data pipelines.
• Hands-on experience building with LLMs including agent loops, tool calling, MCP, or eval harnesses, and intuition about model behavior to reason about training signals.
• Demonstrated ability to own ambiguous, undefined problems end to end and drive them to a shipped system.
• Excellent written and verbal communication; ability to align engineers, researchers, and non-engineering partners on a technical direction.

Responsibilities

• Own the technical foundation for how Scale builds, runs, verifies, and delivers RL environments at scale.
• Design the platform: sandboxed execution, environment packaging/versioning, rollout orchestration, trajectory capture, verifier frameworks, authoring surfaces.
• Ship the hard parts and set technical direction across multiple teams.

Skills

Distributed systems

Python

Containerization

Backend scalability

LLM tooling

Ambiguity handling

Communication

Tools

Docker

VMs

gVisor/Firecracker

Kubernetes

TypeScript/React

Go

Rust

Job description

Scale AI is seeking a Staff Software Engineer, RL Environments, to own the technical foundation for large-scale RL environments. You will design sandboxed execution, packaging/versioning, rollout orchestration, and verifier frameworks, while instrumenting real applications and building task suites to reveal capability gaps.

In this hands-on role, you will lead across teams, write the challenging code, and ensure trustworthy reward signals and throughput for millions of rollouts at scale.