Senior Frontier AI Evaluation Scientist - Equity

turing; United States; Remote; USD 250,000 - 350,000; Full time

Benefits offered by this job

Equity

Publication opportunities

Job summary

Turing is seeking a researcher to advance frontier AI evaluation, training, and improvement. You will tackle problems where benchmarks and datasets are not yet defined, proposing new directions from scratch and publishing in leading venues.

The role emphasizes designing novel benchmarks across STEM domains, generating high-quality synthetic data, and evaluating model reliability, uncertainty, and abstention, with strong autonomy and significant scientific impact.

Qualifications

• PhD or equivalent research experience in a highly technical field (ML, CS, math, physics, chemistry, biology, engineering, or statistics).
• Ability to formulate and execute original research projects.
• Strong grounding in modern large language models and frontier AI research landscape.
• Excellent experimental design and quantitative analysis skills.

Responsibilities

• Identify gaps in benchmark/evaluation literature and design novel benchmarks for STEM domains.
• Develop rigorous task-generation, grading, contamination-control, and validation methodologies to set new evaluation standards.
• Create methods for high-quality synthetic STEM training data and run experiments to separate capability gains from training volume.
• Investigate hallucination, uncertainty, calibration, and epistemic failure; build evaluations for reliability, self‑correction, verification, and abstention.
• Research long-horizon scientific agents and design workflows combining literature search, coding, simulation, tool use, experimentation, and iterative reasoning.
• Pursue new directions in reasoning, model evaluation, AI-for-science, data generation, and emerging capabilities.

Skills

Original research

Quantitative analysis

Experimental design

LLM fundamentals

Education

PhD or equivalent

Job description

Turing is seeking a researcher to advance frontier AI evaluation, training, and improvement. You will tackle problems where benchmarks and datasets are not yet defined, proposing new directions from scratch and publishing in leading venues.

The role emphasizes designing novel benchmarks across STEM domains, generating high-quality synthetic data, and evaluating model reliability, uncertainty, and abstention, with strong autonomy and significant scientific impact.