AI Research Engineer — Human-Centric Evals & Benchmarks

dscout

United States

Remote

USD 250,000 - 400,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Equity program
Built-in bonus

Job summary

Dscout is hiring a research-focused professional to lead human-grounded evaluation for AI models. You will design rubrics, build living benchmarks with real users over long horizons, and publish results that inform product development. A PhD or MS with internships, strong statistics, and Python skills are essential.

Based in the Bay Area with remote-friendly options and an equity program. You will work with a small senior team, collaborate with Head of AI Research, and contribute to papers and

Qualifications

  • Experience in evals, preference learning, or agent evaluation.
  • Strong technical writing and ability to explain work to both researchers and product managers.
  • Hands-on experience building research prototypes and evaluation pipelines.

Responsibilities

  • Design rubrics and evaluation metrics for model judgment.
  • Develop long-horizon benchmarks with real users in context.
  • Publish findings in top venues; co-author with external labs.
  • Collaborate with forward-deployed engineers and delivery teams.

Skills

Published research
PhD or MS with internships
Statistics
Python
LLM evaluation
Experimentation
Technical writing
Bay Area

Education

PhD or MS with internships

Tools

Python

Job description

Dscout is hiring a research-focused professional to lead human-grounded evaluation for AI models. You will design rubrics, build living benchmarks with real users over long horizons, and publish results that inform product development. A PhD or MS with internships, strong statistics, and Python skills are essential.

Based in the Bay Area with remote-friendly options and an equity program. You will work with a small senior team, collaborate with Head of AI Research, and contribute to papers and

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Research Engineer
AI Research Engineer

dscout • United States

Remote
USD 250,000 - 400,000
Equity program
Built-in bonus
AI Researcher
AI Researcher

Vetto • United States

Remote
USD 120,000 - 170,000
Stock option plan
AI Evaluation Researcher — Benchmarks, Signals & Usability
AI Evaluation Researcher — Benchmarks, Signals & Usability

Bison • San Francisco (CA)

On-site
USD 350,000 - 475,000
Health insurance
Dental and vision benefits
Unlimited PTO
+2
Applied AI Researcher - Benchmarking & Task Design
Applied AI Researcher - Benchmarking & Task Design

Raydar • New York (NY)

On-site
USD 160,000 - 200,000
Health coverage
Applied Research Engineer
Applied Research Engineer

Sterling Inspired Staffing. • San Francisco (CA)

On-site
USD 120,000 - 160,000
Comprehensive medical plans
Generous parental leave
Unlimited PTO
+3
Member of Technical Staff, Evaluations
Member of Technical Staff, Evaluations

Bake AI Inc. • San Mateo (CA)

On-site
USD 180,000 - 240,000
AI Evaluation Scientist — Real-World ML Research
AI Evaluation Scientist — Real-World ML Research

Arena Intelligence, Inc. • San Francisco (CA)

On-site
USD 170,000 - 210,000
Competitive compensation
Equity
Health benefits
+2
Research Scientist - AI-Driven Human Insights
Research Scientist - AI-Driven Human Insights

outset • United States

On-site
USD 140,000 - 200,000
Research Engineer, AI Systems & Evaluation
Research Engineer, AI Systems & Evaluation

AItoolnavio • San Francisco (CA)

On-site
USD 140,000 - 210,000
Remote Research Scientist, AI Benchmark Evaluation
Remote Research Scientist, AI Benchmark Evaluation

Ladders • United States

Remote
USD 198,000 - 340,000
Cutting-edge research
Funding for AI research
Open-source contributions
+2