AI Validation Analyst - Backtests & Metrics

present of AI; New York (NY); On-site; USD 120,000 - 180,000; Full time

Job summary

Simile seeks an analyst to verify that AI-driven simulations of human behavior mirror real-world data. You will design validation studies, run backtests in Python, and compare results against ground truth using statistical metrics, clearly describing what the evidence supports and its limits.

The role emphasizes solid statistics, careful data quality judgment, and turning findings into actionable recommendations for modeling and data teams within an AI product context.

Qualifications

• Strong statistics and data quality judgment.
• Ability to translate findings into recommendations for modeling teams.
• Experience validating simulations against real-world data.

Responsibilities

• Design validation studies to compare simulated vs. real data.
• Run backtests in Python and evaluate with statistical metrics.
• Explain evidence and limitations clearly to stakeholders.
• Collaborate with data and modeling teams to improve simulations.

Skills

Python

Statistical analysis

Backtesting

Data quality

Tools

APIs

Python tools

Job description

Simile seeks an analyst to verify that AI-driven simulations of human behavior mirror real-world data. You will design validation studies, run backtests in Python, and compare results against ground truth using statistical metrics, clearly describing what the evidence supports and its limits.

The role emphasizes solid statistics, careful data quality judgment, and turning findings into actionable recommendations for modeling and data teams within an AI product context.