AI Benchmark Problem Designer — Applied Mathematics

Obsidian; San Francisco (CA); Remote; USD 140,000 - 210,000; Full time

Job summary

Obsidian is seeking a Computational Statistics and Applied Mathematics Expert to design graduate-level problems that test AI systems using real scientific software. You will create challenging, reproducible computational tasks, run simulations, interpret results, and refine problems to meet target difficulty.

The role emphasizes deep expertise with specialized computational packages, strong Python skills, and the ability to think strategically about experimental design and data interpretation.

Qualifications

• Graduate-level training in statistics or applied mathematics.
• Proven proficiency with at least one specialized statistical software package.
• Strong Python skills for problem setups and validators.
• Able to work independently and refine problem designs from feedback.
• Comfort with Linux/terminal and remote compute sandboxes.
• Available for 15–20 hours per week.

Responsibilities

• Design original graduate-level computational problems based on real workflows.
• Use specialized packages to craft problems requiring multi-step workflows.
• Test problems against state-of-the-art AI models and refine for target difficulty.
• Plan queries or experiments to uncover information from partial results.

Skills

Python proficiency

Independent thinker

Linux/terminal

Strong communication

Education

MS in statistics or applied mathematics

PhD preferred or 10+ years experience

Tools

rstan

cmdstanr

rjags

brms

nimble

bayesplot

pymc

statsmodels

PyMC3

numPy

SciPy

Job description

Obsidian is seeking a Computational Statistics and Applied Mathematics Expert to design graduate-level problems that test AI systems using real scientific software. You will create challenging, reproducible computational tasks, run simulations, interpret results, and refine problems to meet target difficulty.

The role emphasizes deep expertise with specialized computational packages, strong Python skills, and the ability to think strategically about experimental design and data interpretation.