Model Failure Analytics Scientist

Reinforced Learning; Town of Tonawanda (NY); On-site; USD 70,000 - 120,000; Full time

Job summary

Reinforced Learning in New York is seeking an analytical thinker to help identify where models break and why. You will read large-scale agent rollouts, build a taxonomy of failure, and translate findings into actionable tasks for the next development cycle.

The role requires strong writing, data analysis skills, and the ability to work with Python or SQL to process logs. You will collaborate with researchers and dataset leads to inform customers with precise, well-documented analyses.

Qualifications

• Analytical background in research, data science, investigation or audit.
• Stamina for close reading and the discipline to code it consistently.
• Python or SQL experience for working with logs at scale.
• Strong writing skills.

Responsibilities

• Review model trajectories across domains and categorize failures.
• Develop a shared failure taxonomy with dataset leads and researchers.
• Quantify shifts in failure modes across model versions and training runs.
• Propose tasks targeting persistent failures.
• Prepare analyses for customer-facing reports.

Skills

Analytical mindset

Close reading

Data analysis

Writing

Tools

Python

SQL

Job description

Reinforced Learning in New York is seeking an analytical thinker to help identify where models break and why. You will read large-scale agent rollouts, build a taxonomy of failure, and translate findings into actionable tasks for the next development cycle.

The role requires strong writing, data analysis skills, and the ability to work with Python or SQL to process logs. You will collaborate with researchers and dataset leads to inform customers with precise, well-documented analyses.