Evaluation Scientist: ML & Simulation Analytics

Aaru; New York (NY); On-site; USD 90,000 - 150,000; Full time

Benefits offered by this job

Equity participation

Visa sponsorship

Relocation assistance

Job summary

Aaru is seeking an Evaluation Researcher to own difficult measurement problems at the intersection of machine learning, statistics, behavioral science, and product decision-making. You will define constructs, assemble or create evaluation data, design studies, write analysis and evaluation code, inspect failures, quantify uncertainty, and communicate what the evidence does and does not support.

Successful candidates will work closely with multiple research and product teams in a fast-paced,

Qualifications

• Experience in ML evaluation, statistics, and behavioral science.
• Ability to design rigorous evaluation studies and interpret results.
• A track record of communicating scientific evidence to diverse audiences.

Responsibilities

• Define evaluation constructs and data needs for complex population simulations.
• Design studies including backtests, holdouts, and prospective evaluations.
• Write analysis and evaluation code; inspect failures and quantify uncertainty.
• Produce clear reports distinguishing exploratory findings from evidence used for decisions.
• Collaborate with cross-functional teams to ensure independent measurements.

Skills

ML evaluation

Statistics

Behavioral science

Experiment design

Tools

Python

SQL

Job description

Aaru is seeking an Evaluation Researcher to own difficult measurement problems at the intersection of machine learning, statistics, behavioral science, and product decision-making. You will define constructs, assemble or create evaluation data, design studies, write analysis and evaluation code, inspect failures, quantify uncertainty, and communicate what the evidence does and does not support.

Successful candidates will work closely with multiple research and product teams in a fast-paced,