AI Benchmark Architect for Scientific Computing

Obsidian; San Francisco (CA); Remote; USD 96,000 - 138,000; Part time

Benefits offered by this job

6-week engagement

Part-time 20+ hrs/week

Immediate start

Job summary

Mercor is seeking researchers to author AI evaluation tasks and original, executable problems for frontier models. You will source material, write prompts, and define grading criteria across subdomains with a focus on two areas in mathematics. Engagement is six weeks, part-time, with start date immediate.

Experience with Python or R for scientific computing and familiarity with Git/GitHub and Docker workflows are required to ensure reproducible runs and automated quality checks.

Qualifications

• PhD in mathematics, applied mathematics, or a closely related field.
• Demonstrated depth in at least two of: numerical linear algebra, computational mechanics, computational finance.
• Working proficiency in Python or R for scientific computing.
• Comfortable with GitHub and running code in Docker; PR workflow with automated checks.

Responsibilities

• Source your own material: a published paper, Kaggle dataset, open-source repo, or self-designed scenario.
• Write scientific prompts based on the input.
• Build grading criteria that define a correct answer.
• Calibrate against frontier models; a task ships when strong models fail it more often than succeed.

Skills

Python

R

GitHub

Docker

Education

PhD in mathematics or related field

Tools

GitHub

Docker

Job description

Mercor is seeking researchers to author AI evaluation tasks and original, executable problems for frontier models. You will source material, write prompts, and define grading criteria across subdomains with a focus on two areas in mathematics. Engagement is six weeks, part-time, with start date immediate.

Experience with Python or R for scientific computing and familiarity with Git/GitHub and Docker workflows are required to ensure reproducible runs and automated quality checks.