Evaluation Researcher for Agent-Based Simulations

Aaru; New York (NY); On-site; USD 120,000 - 190,000; Full time

Benefits offered by this job

Competitive salary

Equity

Medical coverage

Vision coverage

Dental coverage

Visa sponsorship

Relocation support

Job summary

Aaru in New York City seeks an Evaluation Researcher to tackle measurement problems at the boundary of ML, statistics, and behavioral science. You will define constructs, assemble data, design studies, write analysis code, and clearly communicate uncertainty.

You will build reusable evaluation Rails and Carts, conduct end-to-end studies, and work with Population Research, Prediction Research, and Deployment while keeping final measurements independent and trustworthy.

Responsibilities

• Own evaluation areas across population construction, predictive systems, individual agent behavior, group dynamics, or end-to-end simulations.
• Design studies using historical backtests, temporal holdouts, prospective outcomes, observational records, controlled experiments, expert judgment, or mixed methods.
• Build reusable evaluation datasets, harnesses, libraries, graders, experiment schemas, leaderboards, and reporting tools.

Job description

Aaru in New York City seeks an Evaluation Researcher to tackle measurement problems at the boundary of ML, statistics, and behavioral science. You will define constructs, assemble data, design studies, write analysis code, and clearly communicate uncertainty.

You will build reusable evaluation Rails and Carts, conduct end-to-end studies, and work with Population Research, Prediction Research, and Deployment while keeping final measurements independent and trustworthy.