Agentic Data Scientist & Benchmarking Researcher

Ifm Us; Sunnyvale (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Institute of Foundation Models (IFM) is seeking a research scientist focused on data and evaluation depth for agentic models. You will own end‑to‑end data pipelines, sourcing high‑quality trajectories and tool‑use data, while building a rigorous evaluation suite to measure progress with reproducible metrics.

You will work with researchers and engineers to ensure data quality, design robust RL environments, and contribute to publications and benchmarks, advancing the state of the art in agentic

Qualifications

• BS, MS or PhD (or equivalent experience) in Computer Science, Machine Learning or a related field.
• 2+ years of experience with a clear emphasis on evaluations and/or training data curation for ML systems.
• Strong Python and PyTorch development experience.
• Demonstrated experience designing and deep‑diving into evaluations, or curating and generating training datasets — ideally both.
• Hands‑on experience using LLM agents in your personal or professional work.
• A habit of reading through raw data and trajectories to understand them and spot issues, and an instinct to distrust a metric until it's validated.

Responsibilities

• Design and run evaluations of agentic capabilities — multi‑step reasoning, tool use, long‑horizon planning, computer use and safety properties — turning ambiguous notions of 'intelligence' into defensible, reproducible metrics.
• Build and harden evaluation harnesses so benchmarks run reliably at scale against training checkpoints, with clear signal on regressions and model health.
• Run experiments characterizing how prompting, sampling, scaffolding and environment design affect agentic performance on internal and public benchmarks.
• Diagnose anomalous eval results mid‑training run — determine whether the cause is the model, the data, the harness or the infrastructure — and communicate the answer clearly.

Skills

Evaluation design

Data curation

LLM agents

Reading raw data

Metric validation

Education

BS/MS/PhD in CS/ML

Tools

Python

PyTorch

RL environments

Job description

Institute of Foundation Models (IFM) is seeking a research scientist focused on data and evaluation depth for agentic models. You will own end‑to‑end data pipelines, sourcing high‑quality trajectories and tool‑use data, while building a rigorous evaluation suite to measure progress with reproducible metrics.

You will work with researchers and engineers to ensure data quality, design robust RL environments, and contribute to publications and benchmarks, advancing the state of the art in agentic