Frontier AI Research Scientist: Benchmarks & Data

Hidden Jobs; United States; Remote; USD 150,000 - 300,000; Full time

Job summary

Hidden Jobs is seeking a researcher to advance frontier AI evaluation within a STEM-focused organization. The role focuses on identifying gaps, designing benchmarks, and creating synthetic data to address open problems in evaluation, reliability, and agentic science.

You will contribute to long-horizon scientific agents that combine literature search, coding, simulation, tool use, and iterative reasoning, with opportunities to publish at leading conferences and influence deployment.

Qualifications

• PhD or equivalent research experience in a highly technical field such as ML, CS, physics, chemistry, biology, engineering, or statistics.
• Demonstrated ability to formulate and execute original research.
• Strong understanding of modern large language models and the frontier AI research landscape.
• Excellent experimental design, analysis, and technical communication skills.

Responsibilities

• Identify high-impact gaps in benchmark and evaluation literature and design novel benchmarks within and across STEM fields.
• Develop evaluation methodologies that handle task generation, grading, contamination control, difficulty calibration, and validation.
• Build methods for generating high-quality synthetic STEM training data and study how task selection, difficulty, diversity, and filtering affect downstream performance.
• Investigate hallucination, uncertainty, calibration, and epistemic failure in technical domains, and develop methods for factual reliability, self-correction, and verification.
• Research long-horizon scientific agents that combine literature search, coding, simulation, tool use, and iterative reasoning.
• Explore new research directions in reasoning, AI-for-science, model evaluation, data generation, and emerging capabilities.

Skills

Original research

Experimental design

Technical communication

Education

PhD or equivalent in a technical field

Job description

Hidden Jobs is seeking a researcher to advance frontier AI evaluation within a STEM-focused organization. The role focuses on identifying gaps, designing benchmarks, and creating synthetic data to address open problems in evaluation, reliability, and agentic science.

You will contribute to long-horizon scientific agents that combine literature search, coding, simulation, tool use, and iterative reasoning, with opportunities to publish at leading conferences and influence deployment.