Physics PhD: AI Benchmarking Scientist for Frontier Models

Obsidian; San Francisco (CA); On-site; USD 60,000 - 120,000; Part time

Job summary

Mercor is hiring PhD and Master's scientists to author AI evaluation tasks for a new benchmark in scientific computing. You will design executable research problems that frontier models currently cannot solve and author prompts based on your sourced material.

Responsibilities include building grading criteria and calibrating against frontier models, with a focus on two or more physics subdomains and strong Python or R computing skills.

Qualifications

• PhD in physics or closely related field.
• Depth in at least two subdomains: condensed matter, optics, quantum information/computing, computational physics, astrophysics, particle physics.
• Proficiency in Python, R, or similar for scientific computing.
• Familiarity with GitHub workflows and Docker for running code.

Responsibilities

• Source your own material: a published paper, a Kaggle dataset, an open-source repo, or a scenario you design
• Write scientific prompts based on the input
• Build the grading criteria that define a correct answer
• Calibrate against frontier models — a task ships only when strong models fail it more often than they succeed

Skills

Python

R

Education

PhD in physics

Tools

GitHub

Docker

Job description

Mercor is hiring PhD and Master's scientists to author AI evaluation tasks for a new benchmark in scientific computing. You will design executable research problems that frontier models currently cannot solve and author prompts based on your sourced material.

Responsibilities include building grading criteria and calibrating against frontier models, with a focus on two or more physics subdomains and strong Python or R computing skills.