RL Environments Engineer

bespokelabs

Mountain View (CA)

On-site

USD 250,000 - 300,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health insurance
Dental coverage
Vision coverage
401(k)
Daily onsite lunch provided
Visa sponsorship
Relocation support

Job summary

Bespoke Labs is seeking an experienced engineer to own coding environments end to end. You will design, build, grade, and harden environments so frontier models must perform real work to pass. The role emphasizes delivering actual tasks at scale, with robust tooling and deterministic grading.

You should have shipped agentic coding environments or tasks and a track record of scaling output through automation, with strong software fundamentals and production-ready code.

Qualifications

  • Strong software engineering fundamentals and production-grade coding experience.
  • Adversarial mindset: anticipate model weaknesses and guard grading.
  • Ownership: build, debug, and ship with minimal supervision.

Responsibilities

  • Build high-fidelity coding worlds around real codebases with real dependencies and mess.
  • Choose which environments are worth building and focus on high-impact tasks.
  • Design tasks across the full lifecycle: prompt, environment, grader, and running models.
  • Ensure grading and sandbox execution are deterministic and hard to game.
  • Create scalable tooling to accelerate the team and reduce friction.

Skills

Software engineering fundamentals
Multiple programming languages
Production code experience
Adversarial mindset
Ownership

Tools

CI/CD
Sandbox environments
Code execution infrastructure

Job description

About Bespoke Labs

Bespoke Labs is an applied AI research lab pioneering data and RL environment curation for training and evaluating agents.

Recently, we curated Open Thoughts , one of the best open reasoning datasets used by multiple frontier labs, trained SOTA specialized models such as Bespoke-MiniChart-7B and Bespoke-MiniCheck , and taught agents to do multi-turn tool-calling with reinforcement learning.

Bespoke is uniquely positioned to capture a large market share of data and RL environment curation.

About the Role

You will own coding environments end to end. You choose what world to build, design the tasks inside it, build the grading, run frontier models against it, and harden it until the only way to pass is to actually do the work.

This is a delivery role. We care most about whether you have done this before and can point to what came out of it. If you have built agentic coding environments or tasks at real volume and can tell us how many and how hard they were, we want to talk.

What You'll Do
  • Build high-fidelity coding worlds around real codebases, with the conventions, dependencies, tooling, and accumulated mess that real software has.
  • Choose which environments are worth building.
  • Targets work where frontier models measurably struggle.
  • Exercises real engineering, meaning navigation, diagnosis, sequencing, and design, and not just writing a function.
  • Rests on a codebase with enough history and structure that shortcuts do not survive.
  • Has a clear pass condition that a reviewer would agree with.
  • Produces many varied tasks from a single world rather than one.
  • Design tasks across the full lifecycle: prompt, environment, grader, running frontier models, failure analysis, and iteration, until each task is rigorous, fair, and hard to game.
  • Build grading and sandboxed execution that is deterministic and cannot be gamed. Assume the model will try to pass without doing the work, and close the door before it finds it.
  • Remove whatever is slowing the team down. Build the internal tooling that makes everyone around you faster.
  • Direct frontier coding agents heavily to build and validate environments, judging their output and catching the quiet failures they produce.
What We're Looking For
  • A record of shipped volume. You have built agentic coding tasks or environments and can show us how many you personally drove and what they cost to produce.
  • Experience scaling that output through automation rather than through more people doing more manual work.
  • Strong software engineering fundamentals and fluency in several languages that holds up in production code.
  • Real experience with production software. Large codebases, build systems, testing, deployment, on-call, and root cause analysis. You know what real engineering work feels like because you have done it.
  • An adversarial mindset. You look at a grader and ask how a model would cheat it, and then you fix that.
  • A clear sense of what frontier coding agents can and cannot do, and where they cut corners.
  • Ownership. You build, debug, and ship without much supervision.
You May Be a Good Fit If You Also
  • Have worked on RL training systems, post-training, verifiers, or tool-use harnesses
  • Come from developer tooling, CI/CD, sandboxes, or code execution infrastructure
  • Have built large-scale automated test generation, fuzzing harnesses, or benchmark suites, which is close cousin work even if it was never called an RL environment
  • Have contributed to a public agentic benchmark such as Terminal-Bench
  • Have open-source work that other people depend on
What We Offer

Location: Mountain View, CA (Onsite)

Base Salary: $250,000 - $300,000 USD / year

Additional Comp: 25% performance-based bonus + equity

Benefits & Perks
  • Health, dental, and vision coverage
  • 401(k)
  • Daily onsite lunch provided
  • Visa sponsorship and relocation support available
  • Direct impact on how the industry trains and evaluates agents

We value different backgrounds and paths into this work. If this role excites you but you do not check every box, apply anyway.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Founding Recruiter
Founding Recruiter

Bespoke Labs • Mountain View (CA)

On-site
USD 150,000 - 225,000
Health, dental, vision
401(k)
Onsite lunch
+3
Member of Technical Staff - Research Engineer, RL Environments
Member of Technical Staff - Research Engineer, RL Environments

Preference Model • San Francisco (CA)

On-site
USD 170,000 - 260,000
Cash & equity compensation
Ownership & autonomy
Top ML engineers
+5
Member of Technical Staff - ML Infrastructure Engineer, RL Post-training
Member of Technical Staff - ML Infrastructure Engineer, RL Post-training

Preference Model, Inc. • San Francisco (CA)

On-site
USD 180,000 - 280,000
Competitive cash & equity ( >90th %ile
Ownership and autonomy
Work alongside senior staff from topML
+5
Staff Software Engineer, RL Environments
Staff Software Engineer, RL Environments

Scale AI, Inc. • San Francisco (CA)

On-site
USD 252,000 - 315,000
Health coverage
Dental coverage
Vision coverage
+4
Staff Software Engineer, RL Environments
Staff Software Engineer, RL Environments

Scale AI, Inc. • New York (NY)

On-site
USD 252,000 - 315,000
Equity
Health benefits
Learning stipend
+1
Senior Software Engineer — RL Environments & Sandboxes
Senior Software Engineer — RL Environments & Sandboxes

Lumos • San Francisco (CA)

On-site
USD 160,000 - 220,000
Founding Research Engineer – RL Environments & Evals
Founding Research Engineer – RL Environments & Evals

Rnb Consultancy • San Francisco (CA)

On-site
USD 150,000 - 200,000
Base salary $150k–$200k
Equity 0.25–0.5%
Profit sharing up to 4% of project
+1
Founding Engineer – RL Environments & Data
Founding Engineer – RL Environments & Data

CGS • San Francisco (CA)

On-site
USD 150,000 - 230,000
Unlimited AI spend
Daily lunch provided
Equity—meaningful
Software Engineer - RL Environments San Francisco
Software Engineer - RL Environments San Francisco

AfterQuery • San Francisco (CA)

On-site
USD 160,000 - 210,000
Equity
Growth opportunities
Applied Researcher - Continual Learning
Applied Researcher - Continual Learning

Prime Intellect AI • San Francisco (CA)

Remote
USD 150,000 - 300,000
Cash compensation: equity incentives
Flexible work options (SF, NY, London,
Visa sponsorship & relocation support
+2