Senior Analyst, Advanced Analytics: Auto Physical Damage (APD)

Liberty Mutual; Boston (MA); Hybrid; USD 83,000 - 157,000; Full time

Job summary

Liberty Mutual is seeking a Senior Analyst, Advanced Analytics (APD) to own model observability across its Auto Physical Damage portfolio. The role balances technical monitoring with collaboration across data scientists, ML engineers, and business partners.

Ideal candidates have 3+ years in analytics, strong SQL/Python skills, and experience with dashboards (Streamlit/Tableau). Hybrid schedule available for proximity to multiple US offices, with remote work otherwise.

Qualifications

• Bachelor's degree in a quantitative field or equivalent experience.
• 3+ years in data analytics, data science, ML engineering, or related analytical role.
• Proficiency in SQL and Python for data manipulation and analysis.
• Solid understanding of machine learning concepts and model performance evaluation.
• Experience building dashboards and reports (Streamlit, Tableau, or similar).
• Strong analytical, problem-solving, and communication skills with attention to detail.

Responsibilities

• Build monitoring infrastructure for production models, including pipelines, dashboards, and alerting.
• Define and track model health metrics for ML and GenAI systems.
• Detect and diagnose issues such as data drift and performance decay; triage to model owners.
• Establish thresholds and alerting; document expected behavior and remediation runbooks.
• Partner with data scientists and ML engineers to integrate monitoring into CI/CD and MLOps/LLMOps lifecycles.
• Support model governance with audit-ready reporting and documentation.
• Analyze production outcomes against KPIs to surface improvement opportunities.

Skills

SQL

Python

Data analytics

Communication

Problem-solving

Attention to detail

Education

Bachelor's degree in a quantitative field

Graduate degree preferred

Tools

Streamlit

Tableau

Excel

PowerPoint

SAS

Emblem

Job description

Senior Analyst, Advanced Analytics: Auto Physical Damage (APD)

Job Locations: US-Remote | US-OH-Columbus | US-WA-Seattle | US-MA-Boston | US-TX-Houston | US-TX-Plano | US-NH-Portsmouth

ID: 2026-77172

Position Type: Full-Time

Job Grade: 15

Department: 043D-13601 Data Sci-Claims & Service

Market: US Retail Markets

Referral Bonus Amount: $0

Minimum Salary: USD $83,000.00/Yr.

Maximum Salary: USD $157,000.00/Yr.

Typical Starting Salary: $97,000 - $130,000

Recruiter: Christina Kacmar

Internal Application Deadline: 8/21/2026

Referral Bonus Eligible?: No

Description

he Auto Physical Damage (APD) Data Science team builds and deploys data science products that power faster, more consistent, and more accurate claims outcomes. Our portfolio spans both traditional machine learning models and Generative AI systems (e.g., document summarization, LLM-driven decision support, and unstructured-data extraction). As our model footprint grows, ensuring these systems remain accurate, reliable, and trustworthy in production is mission-critical.

We are seeking a Model Monitoring Analyst (Senior Analyst, Advanced Analytics) to design, build, and operate the systems that keep our production models healthy. You will be the owner of model observability across the APD portfolio - establishing how we detect performance degradation, data drift, and anomalous behavior for both classical ML and GenAI systems. This is a highly visible role that partners closely with data scientists, ML engineers, claims business partners, and model governance teams.

Candidates who live within 50 miles of Boston, MA; Portsmouth, NH; Seattle, WA; Columbus, OH; or Plano, TX will follow a hybrid schedule, coming into the office two days per week. Otherwise, this role is remote with occasional travel.

About the Team

Key Responsibilities:

• Build monitoring infrastructure for production models, covering both traditional ML and GenAI/LLM systems, including automated pipelines, dashboards, and alerting.
• Define and track model health metrics – for ML: accuracy, precision/recall, AUC, calibration, feature and prediction drift. For GenAI: output quality, hallucination/grounding checks, relevance, latency, token/cost usage, and guardrail adherence.
• Detect and diagnose issues such as data drift, concept drift, performance decay, and data-quality breaks, then triage and escape to the appropriate model owners.
• Establish thresholds and alerting that balance early detection with alert fatigue, and document expected behavior and remediation runbooks.
• Partner with data scientists and ML engineers to integrate monitoring into the model deployment lifecycle (CI/CD, MLOps/LLMOps).
• Support model governance and compliance by producing monitoring evidence, audit-ready reporting, and documentation aligned with enterprise model risk management standards.
• Analyze production outcomes against business KPIs to surface opportunities for model improvement or retraining.
• Communicate findings clearly to both technical and non-technical stakeholders through reporting and periodic model health reviews.

The ideal candidate will have:

• Bachelor's degree in a quantitative field (Statistics, Data Science, Computer Science, Engineering, Economics, or related), or equivalent experience.
• 3+ years of experience in data analytics, data science, ML engineering, or a related analytical role.
• Proficiency in SQL and Python for data manipulation and analysis.
• Solid understanding of machine learning concepts and model performance evaluation.
• Experience building dashboards and reports (e.g., Streamlit, Tableau, or similar).
• Strong analytical, problem-solving, and communication skills, with attention to detail.

Preferred:

• Graduate degree in a quantitative field (Statistics, Data Science, Computer Science, Engineering, Economics, or related), or equivalent experience.
• Experience with model monitoring / observability tooling.
• Experience with A/B testing or experiment design to test impact of solutions.
• Familiarity with GenAI/LLM evaluation concepts – prompt/response quality, hallucination detection, retrieval-augmented generation (RAG), guardrails, and LLM cost/latency monitoring.
• Exposure to cloud platforms (AWS, Azure, or GCP) and MLOps/LLMOps practices.
• Knowledge of the auto claims or insurance domain.

Qualifications

• Bachelor's Degree plus a minimum 3 years, typically 4 or more years of experience, or equivalent, is required.
• Mathematics, Economics, Statistics or other quantitative field are preferred fields of study.
• Advanced knowledge of data sources, tools, statistical principles and methodologies, and techniques.
• Advanced proficiency in Excel (VBA, macros, scripts, formulas, data visualization, etc.), PowerPoint, and statistical software packages (SAS, Emblem).
• Must have good planning, analytical, decision-making and communication skills. Solid understanding of business to improve business outcomes.

Employees may apply for a new role after completing 12 months of employment in their current position.

Employees should review all role requirements and apply only for positions for which they are eligible. Hiring processes may vary by country, including differences in procedures, requirements, and timelines. For country-specific details, please consult your local recruiting / HR team.