Frontier AI Research Scientist: Benchmarks and Evaluation

turing; EE. UU.; A distancia; USD 150.000 - 300.000; Jornada completa

Descripción de la vacante

Turing is seeking a research scientist to work on frontier AI evaluation, data generation, and model reliability. The role targets open problems where benchmarks, datasets, or methodologies do not yet exist, demanding rigorous question formulation and experimental programs.

Responsibilities include designing novel benchmarks across STEM, developing evaluation methodologies, generating synthetic training data, and addressing hallucination and epistemic failure.

Formación

• PhD or equivalent research experience in a highly technical field such as ML, CS, mathematics, physics, chemistry, biology, engineering, or statistics.
• Proven ability to formulate and execute original research.
• Strong understanding of modern large language models and the frontier AI research landscape.
• Excellent experimental design, analysis, and technical communication skills.

Responsabilidades

• Identify high-impact gaps in benchmark and evaluation literature and design novel benchmarks within and across STEM fields.
• Develop evaluation methodologies that handle task generation, grading, contamination control, difficulty calibration, and validation.
• Build methods for generating high-quality synthetic STEM training data and study how task selection, difficulty, diversity, and filtering affect downstream performance.
• Investigate hallucination, uncertainty, calibration, and epistemic failure in technical domains, and develop methods for factual reliability, self-correction, and verification.
• Research long-horizon scientific agents that combine literature search, coding, simulation, tool use, and iterative reasoning.
• Explore new research directions in reasoning, AI-for-science, model evaluation, data generation, and emerging capabilities.

Conocimientos

Original research

Experimental design

Technical communication

Frontier AI/LLMs

Educación

PhD or equivalent in ML/CS/math/physics...

Descripción del empleo

Turing is seeking a research scientist to work on frontier AI evaluation, data generation, and model reliability. The role targets open problems where benchmarks, datasets, or methodologies do not yet exist, demanding rigorous question formulation and experimental programs.

Responsibilities include designing novel benchmarks across STEM, developing evaluation methodologies, generating synthetic training data, and addressing hallucination and epistemic failure.