Frontier AI Research Scientist: Benchmarking

turing; United States; Remote; USD 250,000 - 400,000; Full time

Job summary

Turing is seeking a research-driven scientist to advance how frontier AI systems are evaluated, trained, and improved. You will tackle problems where existing benchmarks fall short, originating new directions from hypothesis through experimentation to publication.

Responsibilities include designing benchmarks, generating synthetic STEM data, and exploring issues like hallucination and calibration to improve factual reliability and self-correction. Publish at leading AI venues.

Qualifications

• PhD or equivalent in a highly technical field.
• Proven ability to formulate and execute original research with substantive contributions.
• Strong understanding of modern large language models and frontier AI research.
• Excellent experimental design, quantitative analysis and technical writing.

Responsibilities

• Identify gaps in benchmark and evaluation literature and propose new research programs.
• Design and build novel benchmarks with rigorous task generation and validation.
• Develop synthetic STEM training data and study effects of selection and filtering.
• Investigate hallucination, calibration and epistemic failure with improved evaluations.
• Research AI systems including long-horizon agents using literature, coding, simulation and tool use.
• Take projects from hypothesis to validated results and publish findings at leading AI venues.

Skills

Original research

Experimental design

Quantitative analysis

Technical writing

LLMs / frontier AI knowledge

Education

PhD or equivalent research experience

Job description

Turing is seeking a research-driven scientist to advance how frontier AI systems are evaluated, trained, and improved. You will tackle problems where existing benchmarks fall short, originating new directions from hypothesis through experimentation to publication.

Responsibilities include designing benchmarks, generating synthetic STEM data, and exploring issues like hallucination and calibration to improve factual reliability and self-correction. Publish at leading AI venues.