Remote Frontier AI Evaluations Researcher

Epoch; United States; Remote; USD 115,000 - 200,000; Full time

Benefits offered by this job

Fully remote environment

30 days PTO per year

Annual staff retreats

Office access in Berkeley, CA

Job summary

Epoch AI seeks a Researcher to lead the evaluation of frontier AI models on real-world, open-ended office tasks. You will curate tasks, design rubrics, and run models to assess performance both quantitatively and qualitatively.

The role emphasizes practical evaluation using AI tools over deep programming, with opportunities across multiple teams. Remote-friendly with generous benefits and flexible work arrangements.

Qualifications

• Analytical thinking demonstrated through rigorous experiments and well-supported findings.
• Grounded mindset to distinguish practical capabilities from hype in AI systems.
• Comfort working with AI agents and delegating tasks in research projects.
• Familiarity with AI benchmarks and evaluations and ability to discuss their value and limits.
• Experience in research and data analysis with light coding to analyze results.
• Strong written communication to convey nuanced observations clearly.

Responsibilities

• Create and curate an evaluation suite of real-world tasks to test AI capabilities.
• Regularly evaluate new AI models and update tasks and rubrics to reflect the landscape.
• Communicate research findings through public reports, blog posts, and data visualizations.
• Conduct data analysis to compare models across tasks.
• Improve processes and automate parts of the evaluation workflow where feasible.

Skills

Analytical thinking

Grounded mindset

Comfort with AI agents & tools

Familiarity with AI benchmarks

Research & data-analysis

Strong written communication

Tools

Python

Job description

Epoch AI seeks a Researcher to lead the evaluation of frontier AI models on real-world, open-ended office tasks. You will curate tasks, design rubrics, and run models to assess performance both quantitatively and qualitatively.

The role emphasizes practical evaluation using AI tools over deep programming, with opportunities across multiple teams. Remote-friendly with generous benefits and flexible work arrangements.