AI Evaluation Scientist Math PhD Frontier Model Benchmark

Obsidian; San Francisco (CA); On-site; USD 96,000 - 179,000; Part time

Job summary

Mercor is hiring PhD and Master's scientists to author AI evaluation tasks (Sci Code). You will author original, executable research problems that today's frontier models cannot solve. The role emphasizes material sourcing, prompt design, and robust grading criteria against leading AI models.

Engagement spans 6 weeks, part-time at 20+ hours per week, with immediate start. You will work with Python or R for scientific computing and use GitHub and Docker in a PR-driven workflow.

Qualifications

• PhD in mathematics, applied mathematics, or closely related field.
• Depth in at least two subdomains: numerical linear algebra, computational mechanics or computational finance.
• Working proficiency in Python or R for scientific computing.
• Comfortable with Git/GitHub and running code in Docker with PR workflows.

Responsibilities

• Source your own material: a published paper, a Kaggle dataset, an open-source repository, or a scenario you design.
• Write scientific prompts based on the input.
• Build the grading criteria that define a correct answer.
• Calibrate against frontier models — a task ships only when strong models fail it more often than they succeed.

Skills

Numerical linear algebra

Computational mechanics

Python or R for scientific computing

Education

PhD in mathematics or related field

Tools

Git/GitHub

Docker

Job description

Mercor is hiring PhD and Master's scientists to author AI evaluation tasks (Sci Code). You will author original, executable research problems that today's frontier models cannot solve. The role emphasizes material sourcing, prompt design, and robust grading criteria against leading AI models.

Engagement spans 6 weeks, part-time at 20+ hours per week, with immediate start. You will work with Python or R for scientific computing and use GitHub and Docker in a PR-driven workflow.