Research Scientist, Agent Evaluation & Benchmarking

Reinforced Learning; Town of Tonawanda (NY); On-site; USD 120,000 - 170,000; Full time

Job summary

Reinforced Learning in the United States seeks a role focused on designing evaluations of agents on long-horizon, tool-using tasks that are statistically sound and diagnostic about why a model failed. You will own held-out evaluation suites and publish benchmarks and methodology.

The candidate should have a research background in evaluation or benchmarking, strong statistics, fluent Python, and clear technical writing to provide contamination controls and failure-mode analysis for customers.

Qualifications

• A research background in evaluation, benchmarking or agents.
• Strong statistics and experimental design.
• Fluent Python and experience with evaluation harnesses.
• Clear technical writing.

Responsibilities

• Held-out evaluation suites across our environment library.
• Metrics beyond pass rate: efficiency, recovery from errors, calibration, safety of actions.
• Contamination controls and leakage detection between training and evaluation.
• Failure-mode analysis and the reports customers act on.
• Published benchmarks and methodology.

Skills

Research background

Statistics & experimental design

Fluent Python

Technical writing

Python

Tools

Evaluation harnesses

Job description

Reinforced Learning in the United States seeks a role focused on designing evaluations of agents on long-horizon, tool-using tasks that are statistically sound and diagnostic about why a model failed. You will own held-out evaluation suites and publish benchmarks and methodology.

The candidate should have a research background in evaluation or benchmarking, strong statistics, fluent Python, and clear technical writing to provide contamination controls and failure-mode analysis for customers.