Remote Research Engineer - Data & Evaluation for RL

Ladders

United States

Remote

USD 150,000 - 350,000

Full time

35 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Remote work
SF in-person option
Visa sponsorship
Relocation assistance
Quarterly team off-sites
Hackathons
Equity incentives
Professional development

Job summary

Ladders is seeking a Research Engineer - Data and Evaluation to join a leader in IT research. The role focuses on designing and scaling data and evaluation systems to measure model performance in complex coding environments.

You will build executable task environments, robust scoring, and investigate failure modes, collaborating with RL researchers to improve data quality and results. Remote position in the United States with compensation ranging from $150,000 to $350,000 annually.

Qualifications

  • Strong Python skills for building data pipelines and evaluation systems.
  • Hands-on experience with language models or agents and their training data.
  • Ability to analyze data and model outputs to identify failure modes.
  • Proven track record of managing technical projects from experimentation to validation.
  • Experience building evaluations or agent environments with debugging and scoring code.

Responsibilities

  • Design and create challenging evaluations for coding and long-horizon agents.
  • Develop executable environments with realistic task setups and clear success criteria.
  • Create and test scoring logic and verifiers, addressing reward hacks and misleading progress measures.
  • Trace model failures through outputs and code to separate model limits from environment issues.
  • Build and scale RL data pipelines that generate and curate complex coding tasks.
  • Collaborate with RL researchers to assess the impact of task and training data changes.

Skills

Python
Data pipelines
Evaluation systems
Language models / agents
Experimentation to validation
Collaboration

Job description

Ladders is seeking a Research Engineer - Data and Evaluation to join a leader in IT research. The role focuses on designing and scaling data and evaluation systems to measure model performance in complex coding environments.

You will build executable task environments, robust scoring, and investigate failure modes, collaborating with RL researchers to improve data quality and results. Remote position in the United States with compensation ranging from $150,000 to $350,000 annually.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Research Engineer - Data and Evaluation
Research Engineer - Data and Evaluation

Ladders • United States

Remote
USD 150,000 - 350,000
Remote work
SF in-person option
Visa sponsorship
+5
Senior Research Engineer — RL Systems & Evaluation (Remote)
Senior Research Engineer — RL Systems & Evaluation (Remote)

24-Mag Llc • New York (NY)

Remote
USD 400,000 - 800,000
Fully remote
Research Engineer | Remote
Research Engineer | Remote

Crossing Hurdles • United States

Remote
USD 400,000 - 800,000
Research Engineer Intern: ML Evaluation & Data Pipelines
Research Engineer Intern: ML Evaluation & Data Pipelines

Thinklance LLP • San Francisco (CA)

On-site
USD 100,000 - 145,000
Meals stipend
Equinox gym membership
Laundry reimbursement
+1
Remote RL Systems Architect
Remote RL Systems Architect

24-MAG • United States

Remote
USD 400,000 - 800,000
Research Engineer - Data and Evaluation (Fully Remote)
Research Engineer - Data and Evaluation (Fully Remote)

Prime Intellect, Inc. • United States

Remote
USD 150,000 - 350,000
Quarterly off-sites
Hackathons
Conferences
+1
Staff Research Engineer — RL, LLMs & Evaluation Systems
Staff Research Engineer — RL, LLMs & Evaluation Systems

hillclimb • San Francisco (CA)

On-site
USD 150,000 - 230,000
Health insurance
Competitive 401k match
Free lunch & dinner
Senior AI Engineer — RL & LLM Evaluation Lead
Senior AI Engineer — RL & LLM Evaluation Lead

EnableComp LLC • United States

Remote
USD 120,000 - 180,000
Remote Research Engineer: High-Performance AI Inference
Remote Research Engineer: High-Performance AI Inference

Ladders • United States

Remote
USD 150,000 - 350,000
Remote work flexibility
Visa sponsorship
Relocation assistance
+4
Senior AI Engineer – RLHF & Model Evaluation (Remote)
Senior AI Engineer – RLHF & Model Evaluation (Remote)

prolificacademicltd • United States

Remote
USD 91,000 - 129,000
Fully remote
Flexible scheduling
Short onboarding
+1