Physics AI Benchmark Scientist: Prompt Design & Evaluation

Mercor; Greater London; Remote; GBP 11,000 - 18,000; Full time

Job summary

Mercor is seeking PhD or Master’s scientists to author AI evaluation tasks for Sci Code in collaboration with leading AI labs on a new benchmark for scientific computing. You will source material, design executable research problems, and craft scoring criteria to challenge frontier models.

The role requires deep knowledge in at least two physics subdomains, strong Python or R skills, and experience with Git/GitHub and Docker workflows. This six-week, part-time engagement starts immediately.

Qualifications

• PhD in physics, applied physics, or closely related field.
• Depth in at least two subdomains: condensed matter, optics, quantum information/computing, computational physics, astrophysics, or particle physics.
• Proficiency in Python, R, or another language for scientific computing.
• Comfortable with Git/GitHub and running code in Docker with PR workflow and automated checks.

Responsibilities

• Source your own material: published paper, Kaggle dataset, open-source repo, or self-designed scenario.
• Write scientific prompts based on the input.
• Build the grading criteria defining correct answers.
• Calibrate against frontier models so a task is released only when models struggle.

Skills

Python

R

Git/GitHub

Docker

Education

PhD in physics or closely related field

Tools

Git/GitHub

Docker

Job description

Mercor is seeking PhD or Master’s scientists to author AI evaluation tasks for Sci Code in collaboration with leading AI labs on a new benchmark for scientific computing. You will source material, design executable research problems, and craft scoring criteria to challenge frontier models.

The role requires deep knowledge in at least two physics subdomains, strong Python or R skills, and experience with Git/GitHub and Docker workflows. This six-week, part-time engagement starts immediately.