AI Benchmark Designer: Computational Statistics

Mercor; San Francisco (CA); On-site; USD 83,000 - 165,000; Part time

Job summary

Mercor is building a large-scale benchmark to test how well AI systems solve advanced scientific problems. You will design graduate-level computational challenges that use real software to run simulations, interpret results, and design experiments.

The role emphasizes puzzle-design thinking over brute-force computation and requires strong Python skills. You will craft problems leveraging specialized packages in R/Python and work with Linux-based compute sandboxes, contributing to a challenging

Qualifications

• Graduate-level training in statistics, applied mathematics, or a closely related quantitative field.
• Proven proficiency with at least one specialized software package, demonstrated via research or professional work.
• Strong Python skills—writing problem setups, oracle functions, and solution validators.
• Ability to work independently and refine problem designs based on feedback.
• Comfortable in a Linux/terminal environment with remote compute sandboxes.
• Available for 15–20 hours per week.

Responsibilities

• Design original problems requiring skilled use of statistical, mathematical, or scientific software.
• Create and test problems against AI models and refine difficulty.
• Plan problems that require multi-step workflows and strategic experimentation.

Skills

Strong Python skills

Independent work

Linux/terminal proficiency

Part-time availability 15–20 hours

Education

MS or PhD preferred; PhD preferred, or MS with 10+ years of experience

Tools

rstan

cmdstanr

brms

lavaan

deSolve

KFAS

lme4

mgcv

spatstat

mclust

Job description

Mercor is building a large-scale benchmark to test how well AI systems solve advanced scientific problems. You will design graduate-level computational challenges that use real software to run simulations, interpret results, and design experiments.

The role emphasizes puzzle-design thinking over brute-force computation and requires strong Python skills. You will craft problems leveraging specialized packages in R/Python and work with Linux-based compute sandboxes, contributing to a challenging