AI Benchmark Architect for Scientific Computing

Mercor; Toronto; Remote; CAD 83,000 - 165,000; Part time

Job summary

Mercor is hiring PhD and Master's scientists to author AI evaluation tasks (Sci Code) for a new benchmark in scientific computing. You will craft original, executable research problems that frontier models currently struggle to solve.

Domains require depth in at least two subfields with a coding focus; Python or R for scientific computing is essential. The role involves sourcing material, writing prompts, and building robust grading criteria for model evaluation.

Qualifications

• PhD in mathematics or closely related field required.
• Demonstrated depth in at least two subdomains (e.g., numerical linear algebra, computational mechanics, computational finance).
• Proficiency in Python or R for scientific computing and ability to run code in Docker with PR workflows.

Responsibilities

• Source your own material: published paper, Kaggle dataset, open-source repo, or a scenario you design.
• Write scientific prompts based on the input and design executable research problems.
• Build the grading criteria defining a correct answer and calibrate against frontier models.

Skills

Python

R

Scientific computing

Git/GitHub

Docker

Mathematics depth

Education

PhD in mathematics

Master's degree in related field

Tools

GitHub

Docker

Job description

Mercor is hiring PhD and Master's scientists to author AI evaluation tasks (Sci Code) for a new benchmark in scientific computing. You will craft original, executable research problems that frontier models currently struggle to solve.

Domains require depth in at least two subfields with a coding focus; Python or R for scientific computing is essential. The role involves sourcing material, writing prompts, and building robust grading criteria for model evaluation.