Model Monitoring Analyst – Auto Damage Analytics

Liberty Mutual Insurance; Columbus (OH); Hybrid; USD 90,000 - 130,000; Full time

Job summary

Liberty Mutual Insurance is seeking a Model Monitoring Analyst to design, build, and operate observability systems for APD data science models, including traditional ML and GenAI. You will own health metrics, drift detection, and alerting across production models.

The role partners with data scientists, ML engineers, and governance teams. The position requires SQL and Python expertise, dashboard development, and strong communication skills.

Qualifications

• Bachelor's degree in a quantitative field or equivalent experience.
• 3+ years in data analytics, data science, ML engineering, or related role.
• Proficiency in SQL and Python for data manipulation and analysis.
• Solid understanding of machine learning concepts and model performance evaluation.
• Experience building dashboards/reports (e.g., Streamlit, Tableau, or similar).
• Strong analytical, problem-solving, and communication skills.

Responsibilities

• Build monitoring infrastructure for production models including pipelines, dashboards, and alerting.
• Define and track model health metrics for ML and GenAI systems.
• Detect and diagnose data drift, concept drift, and performance decay; triage issues to model owners.
• Establish thresholds and alerting with balanced sensitivity; document runbooks.

Skills

SQL

Python

Statistical knowledge

Analytical thinking

Communication skills

Education

Bachelor's degree

Graduate degree

Tools

Excel

PowerPoint

SAS

Emblem

Job description

Liberty Mutual Insurance is seeking a Model Monitoring Analyst to design, build, and operate observability systems for APD data science models, including traditional ML and GenAI. You will own health metrics, drift detection, and alerting across production models.

The role partners with data scientists, ML engineers, and governance teams. The position requires SQL and Python expertise, dashboard development, and strong communication skills.