RL & Data Benchmarking Researcher

protege; United States; Remote; USD 150,000 - 230,000; Full time

Job summary

Protege is seeking a Machine Learning Researcher focused on reinforcement learning and agentic systems to help define, design, and evaluate high-value datasets and benchmarks used to assess advanced AI systems.

In this role you’ll work closely with research and engineering teams to translate real-world workflows into structured tasks, environments, and evaluation assets, driving rigorous data-centric AI evaluation at DataLab.

Qualifications

• PhD or equivalent Master’s Degree + 4+ years industry experience in ML or related fields.
• Strong understanding of AI model training pipelines and the role of data in shaping model performance.
• Experience with large, unstructured or semi-structured datasets used to train/evaluate ML systems.
• Experience with reinforcement learning, agentic systems, and multi-step model evaluation.
• Experience designing tasks, benchmarks, environments, simulations, or evaluation frameworks for real-world model behavior.

Responsibilities

• Design and build datasets, tasks, environments for benchmarking agentic systems.
• Translate real-world workflows into structured tasks, interactions, trajectories, and verifiable outcomes.
• Develop frameworks that assess diversity, realism, coverage, fidelity, informativeness, and downstream usefulness of datasets.
• Benchmark model behavior in RL-style or agentic settings and connect failures to data/design gaps.
• Build scalable evaluation and validation tooling and improve reproducible experimentation infrastructure.
• Collaborate across research, engineering, and product to improve data evaluation methodologies and practices.

Skills

Reinforcement learning

Agentic systems

Evaluation benchmarking

Data science

Experiment design

Education

PhD or equivalent Master's Degree + 4+ years industry experience

Job description

Protege is seeking a Machine Learning Researcher focused on reinforcement learning and agentic systems to help define, design, and evaluate high-value datasets and benchmarks used to assess advanced AI systems.

In this role you’ll work closely with research and engineering teams to translate real-world workflows into structured tasks, environments, and evaluation assets, driving rigorous data-centric AI evaluation at DataLab.