Model Monitoring Analyst – Advanced Analytics (APD)

Embedded Shishya; Boston (MA); On-site; USD 110,000 - 170,000; Full time

Job summary

Liberty Mutual's Auto Physical Damage Data Science team builds and deploys data science products that power faster, more accurate claims outcomes. We seek a Model Monitoring Analyst to own observability across the APD portfolio, establishing how we detect performance degradation, data drift, and anomalous behavior for both classical ML and GenAI systems.

The role is hybrid for candidates within 50 miles of Boston, MA; otherwise, it is remote with travel.

Qualifications

• Bachelor's degree in a quantitative field or equivalent experience.
• 3+ years in data analytics, data science, ML engineering, or related roles.
• Proficiency in SQL and Python for data manipulation and analysis.
• Solid understanding of ML concepts and model evaluation.
• Experience building dashboards/reports (Streamlit, Tableau).
• Graduate degree in a quantitative field or equivalent experience.

Responsibilities

• Build monitoring infrastructure for production models including pipelines, dashboards, and alerting.
• Define and track model health metrics for ML and GenAI systems.
• Detect data drift, concept drift, performance decay, and data-quality issues.
• Establish thresholds and alerting; document remediation runbooks.
• Integrate monitoring into CI/CD and MLOps/LLMOps workflows.
• Support governance with audit-ready monitoring evidence and reporting.
• Analyze production outcomes against KPIs and surface improvement opportunities.
• Communicate findings to technical and non-technical stakeholders.

Education

Bachelor's degree in a quantitative field

Graduate degree in a quantitative field

Tools

SQL

Python

Streamlit

Tableau

Job description

Liberty Mutual's Auto Physical Damage Data Science team builds and deploys data science products that power faster, more accurate claims outcomes. We seek a Model Monitoring Analyst to own observability across the APD portfolio, establishing how we detect performance degradation, data drift, and anomalous behavior for both classical ML and GenAI systems.

The role is hybrid for candidates within 50 miles of Boston, MA; otherwise, it is remote with travel.