Senior AI Model Evaluation Analyst

Elsevier; New York (NY); On-site; USD 115,000 - 210,000; Full time

Benefits offered by this job

Health plan

Disability insurance

Dependent Care

Life Insurance

Retirement Plan

Family benefits

Job summary

Elsevier is seeking a Senior Data Analyst III to join our generative AI evaluation team. You will help evaluate AI models, analyze performance metrics, and ensure outputs meet healthcare standards.

This role focuses on rigorous validation, fairness, and clinical relevance within healthcare decision-support tools. You will collaborate with AI researchers, product managers, clinicians, and engineers to drive improvements in model evaluation methodologies and analytics roadmaps across healthcare

Qualifications

• 6-9 years of relevant data analytics experience.
• Bachelor's degree required; Master's or PhD preferred in Mathematics/Science.
• Demonstrated analytics strategy leadership and enterprise project experience.
• Experience in data analytics roles.
• Expertise in building evaluation frameworks.
• Strong SQL (SQL Server, Snowflake, PostgreSQL) skills.
• Proficiency with BI tools (Tableau or PowerBI).
• Python for analysis.

Responsibilities

• Lead analytics frameworks for evaluating generative AI models in healthcare.
• Define strategic direction for AI evaluation methodologies in product area.
• Manage analytics roadmap for model evaluation.
• Collaborate with stakeholders to manage timelines and deliverables.
• Develop queries and metrics to assess AI performance and output quality.
• Lead data model creation and maintenance for evaluation frameworks and monitoring.
• Conduct QA of AI outputs and participate in risk analysis and mitigation.
• Understand HIPAA, PHI/PII handling in AI contexts.

Job description

Elsevier is seeking a Senior Data Analyst III to join our generative AI evaluation team. You will help evaluate AI models, analyze performance metrics, and ensure outputs meet healthcare standards.

This role focuses on rigorous validation, fairness, and clinical relevance within healthcare decision-support tools. You will collaborate with AI researchers, product managers, clinicians, and engineers to drive improvements in model evaluation methodologies and analytics roadmaps across healthcare