Evaluation Researcher — ML & Behavioral Systems

Aaru Inc.; New York (NY); Hybrid; USD 200,000 - 600,000; Full time

Benefits offered by this job

Equity ownership

Visa sponsorship

Relocation support

Comprehensive health benefits

Job summary

Aaru Inc. is seeking an in-person Evaluation Researcher to join our NYC team. You will own measurement problems at the intersection of machine learning, statistics, and behavioral science, designing studies and writing analysis to guide product decisions.

You will work with cross-functional teams to build end-to-end evaluations, diagnose failures, and communicate uncertainty clearly to stakeholders. The role emphasizes rigor, independence, and impactful findings.

Qualifications

• Design studies using historical backtests, holdouts, or prospective outcomes.
• Quantify uncertainty and interpret results for product decisions.
• Build evaluation datasets and reusable infrastructure.

Responsibilities

• Own evaluation areas across population construction, predictive systems, and end-to-end simulations.
• Design studies and write analysis to answer key questions about realism, calibration, and decision quality.
• Develop reusable diagnostics to test profile and population coherence, and compare simulations with real outcomes.

Skills

Experimental design

Statistical analysis

ML evaluation

Python

Communication

Tools

Python

SQL

R

Pandas

Job description

Aaru Inc. is seeking an in-person Evaluation Researcher to join our NYC team. You will own measurement problems at the intersection of machine learning, statistics, and behavioral science, designing studies and writing analysis to guide product decisions.

You will work with cross-functional teams to build end-to-end evaluations, diagnose failures, and communicate uncertainty clearly to stakeholders. The role emphasizes rigor, independence, and impactful findings.