Principal Clinical AI Evaluation Specialist - MN Preferred

UnitedHealth Group; Minnetonka (MN); Hybrid; USD 113,000 - 193,000; Full time

Benefits offered by this job

Benefits package

Incentive programs

Equity and 401k

Job summary

UnitedHealth Group is seeking a Principal-level AI governance expert to lead evaluation frameworks for clinical AI use cases and vendor vetting. You will ensure rigorous evidence, supporting Council decisions with robust tests, metrics, and risk assessments.

The role spans cross-functional collaboration across clinical, data science, engineering, Legal, and Compliance, with remote work options nationwide and required in-office presence in specified hubs.

Qualifications

• 5+ years of experience working with healthcare data, healthcare technology, clinical operations, healthcare analytics, or related healthcare domains.
• 5+ years of experience using Python, SQL, R, or similar tools to develop, test, evaluate, or validate analytic, machine learning, or AI-enabled solutions.
• 5+ years of experience working across clinical, technical, operational, data science, or product teams to translate complex technical concepts into business or clinical recommendations.
• 3+ years of hands-on experience developing, implementing, testing, or evaluating AI solutions, including predictive models, NLP systems, LLM-based applications, chatbots, retrieval-augmented generation (RAG) systems, or agentic workflows.
• 3+ years of experience designing, conducting, or overseeing testing, validation, benchmarking, or performance assessment of AI-enabled solutions, including evaluation of accuracy, safety, reliability, bias, limitations, or clinical appropriateness.

Responsibilities

• Partner with Council leadership and program consultants to apply the submission framework to each use case, ensuring analytic components align the AI's intended use and defined risks, metrics, and thresholds.
• Define use case-specific measurable performance, success criteria, translating clinical risks into measurable metrics and thresholds for decision-making.
• Ensure complete and clearly defined evaluation methods at intake, including data, populations, comparators, metrics, test methods, and study design.
• Serve as the primary technical liaison across data science, clinical, and engineering SMEs, synthesizing inputs into a consistent evaluation approach.
• Ensure that evaluation results support clear decisions against defined performance and success thresholds.
• Develop concise, decision-ready summaries of performance against defined thresholds, including key limitations of the AI supported program and AI evaluation methodology.
• Serve as a senior methodological resource during Council review, addressing questions on evaluation design and results.
• Integrate analytic, clinical, and operational inputs into a single, aligned recommendation.
• Define requirements for post-deployment monitoring, including metrics, thresholds, accountable governance groups, and triggers for re-review.
• Design and execute analytic evaluations of clinical AI use cases using real-world healthcare data.
• Perform hands-on data programming, analysis, and interpretation to support Council reviews and broader evaluation initiatives

Skills

Python

SQL

R

AI/ML

NLP

Tools

Git

Data Visualization

Job description

At UnitedHealthcare, we're simplifying the health care experience, creating healthier communities and removing barriers to quality care. The work you do here impacts the lives of millions of people for the better. Come build the health care system of tomorrow, making it more responsive, affordable and optimized. Ready to make a difference? Join us to start Caring. Connecting. Growing together.

This Principal-level role serves as a senior technical evaluation and AI governance expert within a multidisciplinary team supporting the UHC Clinical AI Governance Council. This individual contributor role is responsible for ensuring that AI-enabled clinical products and use cases are appropriately tested, evaluated, and documented prior to implementation and that evidence presented to the Council is rigorous, reproducible, and aligned with clinical risks, performance metrics, AI safety considerations, and defined success thresholds.

The position owns the evaluation framework and evidence standards used to support AI vendor vetting and clinical AI review. The role ensures that each use case is evaluated against clearly defined risks, metrics, and thresholds to enable objective and consistent Council decision-making. The role advises on and reviews evaluation strategies for clinical AI use cases and, when needed, conducts independent assessments to validate performance claims, testing results, and evidence submitted for Council review.

The role serves as the primary technical and methodological liaison among Council leadership, program consultants, and subject matter experts across clinical, data science, engineering, Legal, and Compliance domains. The position provides independent evaluation of AI-enabled solutions and translates complex AI concepts and evidence into clear, decision-ready recommendations for senior clinical leadership.

You'll enjoy the flexibility to work remotely* from anywhere within the U.S. as you take on some tough challenges. For all hires in the Minneapolis or Washington, D.C. area, you will be required to work in the office a minimum of four days per week.

Primary Responsibilities:

• Partner with Council leadership and program consultants to apply the submission framework to each use case, ensuring analytic components align the AI's intended use and defined risks, metrics, and thresholds
• Define use case-specific measurable performance, success criteria, translating clinical risks into measurable metrics and thresholds for decision-making
• Ensure complete and clearly defined evaluation methods at intake, including data, populations, comparators, metrics, test methods, and study design
• Serve as the primary technical liaison across data science, clinical, and engineering SMEs, synthesizing inputs into a consistent evaluation approach
• Ensure that evaluation results support clear decisions against defined performance and success thresholds
• Develop concise, decision-ready summaries of performance against defined thresholds, including key limitations of the AI supported program and AI evaluation methodology
• Serve as a senior methodological resource during Council review, addressing questions on evaluation design and results
• Integrate analytic, clinical, and operational inputs into a single, aligned recommendation
• Define requirements for post-deployment monitoring, including metrics, thresholds, accountable governance groups, and triggers for re-review
• Design and execute analytic evaluations of clinical AI use cases using real-world healthcare data
• Perform hands-on data programming, analysis, and interpretation to support Council reviews and broader evaluation initiatives

Required Qualifications:

• 5+ years of experience working with healthcare data, healthcare technology, clinical operations, healthcare analytics, or related healthcare domains
• 5+ years of experience using Python, SQL, R, or similar tools to develop, test, evaluate, or validate analytic, machine learning, or AI-enabled solutions
• 5+ years of experience working across clinical, technical, operational, data science, or product teams to translate complex technical concepts into business or clinical recommendations
• 3+ years of hands-on experience developing, implementing, testing, or evaluating AI solutions, including predictive models, NLP systems, LLM-based applications, chatbots, retrieval-augmented generation (RAG) systems, or agentic workflows
• 3+ years of experience designing, conducting, or overseeing testing, validation, benchmarking, or performance assessment of AI-enabled solutions, including evaluation of accuracy, safety, reliability, bias, limitations, or clinical appropriateness

Preferred Qualifications:

• Experience developing or deploying LLM-based applications, chatbots, retrieval-augmented generation (RAG) systems, agentic AI workflows, or AI-enabled clinical decision-support tools
• Experience evaluating AI solutions in healthcare settings, including assessment of accuracy, safety, bias, reliability, workflow integration, or clinical appropriateness
• Experience supporting AI governance, model risk management, responsible AI, algorithm oversight, or technology review committees
• Experience conducting vendor assessments, proof-of-concept testing, pilot evaluations, or technical due diligence for AI-enabled products
• Experience presenting complex technical findings to senior leaders, clinicians, or governance bodies and translating results into actionable recommendations
• Experience reviewing, designing, or overseeing test plans for AI-enabled products in healthcare or other regulated environments

*All employees working remotely will be required to adhere to UnitedHealth Group's Telecommuter Policy

Pay is based on several factors including but not limited to local labor markets, education, work experience, certifications, etc. In addition to your salary, we offer benefits such as,

• a comprehensive benefits package
• incentive and recognition programs
• equity stock purchase and 401k contribution (all benefits are subject to eligibility requirements)

No matter where or when you begin a career with us, you'll find a far-reaching choice of benefits and incentives. The salary for this role will range from $112,700 - $193,200 annually based on full-time employment. We comply with all minimum wage laws as applicable.

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission. #UHCPJ

UnitedHealth Group is an Equal Employment Opportunity employer under applicable law and qualified applicants will receive consideration for employment without regard to race, national origin, religion, age, color, sex, sexual orientation, gender identity, disability, or protected veteran status, or any other characteristic protected by local, state, or federal laws, rules, or regulations.

UnitedHealth Group is a drug - free workplace. Candidates are required to pass a drug test before beginning employment.