Software Engineer, RL Data

Free resume; San Francisco (CA); Hybrid; USD 30,000 - 50,000; Full time

Job summary

SpaceXAI is hiring a Software Engineer for the RL Data team to shape training data and evaluation for agents. You will build environments that mimic production codebases and design graders to evaluate open-ended work, turning messy problems into scalable, verifiable data pipelines.

You’ll work on failing-proof data generation through automated checks and robust data production systems, with emphasis on reliability and reproducibility in a remote, full-time role.

Qualifications

• Strong software engineering fundamentals.
• Experience building non-trivial systems in Python (data pipelines, test/eval infrastructure, sandboxes, or backend services).

Responsibilities

• Turning a capability gap into thousands of verifiable tasks for the agent to root-cause.
• Building environments that resemble production with real dependencies and realistic apps.
• Designing graders for open-ended work that distinguish good solutions without unfairly penalizing valid approaches.
• Building automated checks to catch reward hacking, leaked answers, and broken environments before data reaches training.
• Scaling from thousands to millions of tasks with trusted filters and judges.

Skills

Python

Data pipelines

Eval infra

Backend services

Sandbox environments

Job description

Direct company application on jobs.ashbyhq.com . Zero third-party aggregator forms.

Job Description

Our mission is to automate coding. The first step in our journey is to build the best tool for professional programmers, using a combination of inventive research, design, and engineering. Our organization is very flat, and our team is small and talent dense. We particularly like people who are truth-seeking, passionate, and creative. We enjoy spirited debate, crazy ideas, and shipping code.

Software Engineer, RL Data

About the Role

The data behind a model shapes it more than almost anything else. On the RL Data team at SpaceXAI, you decide what that data is for our agents: which problems they practice, what world they practice in, and how they're judged. You'll take a weakness, like an agent that gives up on hard bugs or misuses a calendar API, and turn it into training data that fixes it, then show the next model is measurably better.

You form hypotheses about why data is helping or misleading the model, build the systems to test them, run ablations, and read the rollouts to find the bug, shortcut or mismatch behind a result. Every flaw you catch, or miss, shows up in what the next Grok models learn.

What you’ll work on

Turning a capability gap into thousands of verifiable tasks - for example, planting subtle bugs deep in a real project's dependencies for the agent to root-cause.

Building environments that behave like production: real codebases with their real dependencies, and simulations of apps like email, calendar, and docs that match the real ones down to search, errors, and pagination.

Designing graders for open-ended work that separate good solutions from bad without penalizing valid alternatives or rewarding shortcuts.

Building automated checks that catch reward hacking, leaked answers, broken environments, and paths the real product wouldn't allow, before data reaches training.

Scaling from a thousand hand-checked tasks to millions, with filters and judges you can trust in place of a person reading each one

You may be a fit if

You have strong software engineering fundamentals, and you've built non-trivial systems in Python: data pipelines, test or eval infrastructure, sandboxes, or backend services.

You have taste for what makes an environment trustworthy: faithful enough that skills transfer, strict enough that shortcuts fail, and simple enough to debug when a result looks wrong.

You are careful and systematic - understanding both aggregate metrics and close reading of raw outputs.

You enjoy turning messy, open-ended problems into repeatable, high-quality data production systems.

RL or ML experience is a plus but not required. We welcome strong engineers who want to do research on what makes models better.

Remote full-time en 30,000-50,000 USD/yr