Computational Statistics Benchmark Designer

Mercor; San Francisco (CA); On-site; USD 120,000 - 150,000; Full time

Job summary

Obsidian in San Francisco seeks a Computational Statistics and Applied Mathematics Expert to design benchmark problems that test AI systems' ability to solve real scientific workflows. You will craft graduate-level problems requiring advanced software, run simulations, interpret results, and refine tasks against state-of-the-art models.

You will leverage R, Python, and specialized packages to create reproducible challenges, prioritize robust reasoning over brute force, and contribute to scalable

Qualifications

• Graduate-level training in statistics or applied mathematics, or equivalent research experience.
• Proven proficiency with at least one specialized statistical, mathematical, or scientific software package.
• Strong Python skills for problem setups, oracle functions, and validators.

Responsibilities

• Design original benchmark problems using specialized computational packages.
• Run simulations and interpret results within defined scientific workflows.
• Refine problems based on testing against state-of-the-art AI models.
• Ensure problem sets are reproducible and scalable.

Skills

Python

Statistics

Applied mathematics

Linux/terminal

Education

MS or PhD in statistics or applied mathematics

10+ years of research experience

Tools

Stan (rstan)

cmdstanr

rjags

brms

Job description

Obsidian in San Francisco seeks a Computational Statistics and Applied Mathematics Expert to design benchmark problems that test AI systems' ability to solve real scientific workflows. You will craft graduate-level problems requiring advanced software, run simulations, interpret results, and refine tasks against state-of-the-art models.

You will leverage R, Python, and specialized packages to create reproducible challenges, prioritize robust reasoning over brute force, and contribute to scalable