AI Benchmark Designer: Advanced Statistical Challenges

Mercor

Los Angeles (CA)

On-site

USD 120,000 - 180,000

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

Mercor is seeking a Computational Statistics and Applied Mathematics Expert to design challenging graduate-level problems that require using specialized statistical and scientific software. You will test problems against cutting-edge AI models, refine them through iterative evaluation, and build tasks that require strategic planning and multi-step workflows.

The role emphasizes deep domain expertise in statistics or applied mathematics, with hands-on experience using packages like rstan, brms,

Qualifications

Graduate-level training in statistics, applied mathematics, or a closely related quantitative field.
Proven proficiency with at least one specialized statistical, mathematical, or scientific software package, demonstrated through research publications, open-source contributions, or professional work.
Strong Python skills — you'll be writing problem setups, oracle functions, and solution validators

Responsibilities

Design original graduate-level problems that require skilled use of specialized statistical, mathematical, or scientific software packages.
Test problems against state-of-the-art AI models and refine them to hit target difficulty.
Plan sequences of queries or experiments to uncover information not directly visible and read partial results strategically.
Refine problem designs through multiple iterations to ensure rigorous evaluation of AI systems.