Principal AI/ML Engineer - Post Deployment Governance

Mayo Foundation for Medical Education and Research; Rochester (MN); Hybrid; USD 163,000 - 237,000; Full time

Benefits offered by this job

Medical plan options

Dental coverage

Vision plan

Pre-Tax Savings (HSA/FSAs)

Competitive retirement package

Job summary

Mayo Clinic is seeking a Principal AI/ML Engineer — Post-Deployment Governance to serve as the enterprise technical authority for monitoring, reporting, and governance after deployment. You will shape risk-based AVM/ PDRS standards and ensure evidence-based decisions across healthcare AI deployments.

You will apply data science, AI/ML engineering, and regulatory insight to validate metrics, evidence, and thresholds while mentoring engineers and aligning with cross-functional teams.

Qualifications

• Advanced AI/ML engineering experience in healthcare or regulated settings.
• Leadership experience managing complex projects and cross-functional teams.
• Strong communication to non-technical stakeholders and executive audiences.
• Familiarity with cloud environments, data governance, and regulatory requirements.
• Experience mentoring engineers and Product Leads.

Responsibilities

• Provide strategic and technical leadership for enterprise post-deployment governance, measurement, monitoring and reporting.
• Define risk-proportionate requirements across pilots, full implementation, and post-deployment changes.
• Establish standards for signals, metrics, baselines, targets, thresholds, and uncertainty.
• Define monitoring-readiness for sources, data cards, handoffs, and ownership.
• Review governance content and advise on policy alignment and evidence sufficiency.
• Apply data science and AI engineering methods to assess metric validity and conclusions.
• Review observability and telemetry with version context and change detection.
• Lead escalation on complex governance questions and provide interim controls.
• Develop templates, rubrics, and training for governance processes.
• Coordinate with product teams, vendors, and enterprise groups on methods and ownership.

Skills

AI/ML leadership

Healthcare domain knowledge

Cloud infrastructure

Governance and compliance

Mentoring/communication

Education

Master’s degree with 7+ years

Bachelor’s degree with 9+ years

Ph.D. preferred

Job description

Why Mayo Clinic

Mayo Clinic is top-ranked in more specialties than any other care provider according to U.S. News & World Report. As we work together to put the needs of the patient first, we are also dedicated to our employees, investing in competitive compensation and comprehensive benefit plans– to take care of you and your family, now and in the future. And with continuing education and advancement opportunities at every turn, you can build a long, successful career with Mayo Clinic.

Benefits Highlights

• Medical:Multiple plan options.
• Dental:Delta Dental or reimbursement account for flexible coverage.
• Vision:Affordable plan with national network.
• Pre-Tax Savings:HSA and FSAs for eligible expenses.
• Retirement: Competitive retirement package to secure your future.

Responsibilities

As the Principal AI/ML Engineer — Post-Deployment Governance within AI Validation & Monitoring (AVM), you will serve as the enterprise technical and methodological authority for post-deployment monitoring and reporting, measurement, lifecycle evidence, and Post Deployment Monitoring (PDM) and Post Deployment Reporting Summary (PDRS) governance. You will define risk-proportionate AIA Governance requirements and standards for monitoring readiness; performance and functionality; patient safety; adoption and fidelity; outcomes; change and retesting; metrics, formulas, baselines, targets, and thresholds; subgroup interpretation; uncertainty; and evidence confidence. You will apply data science, AI/ML engineering, statistical, and systems expertise to determine whether evidence is traceable, appropriately interpreted, proportionate to risk, and decision-ready.

Within AIA Governance, you will review drafted monitoring, reporting, measurement, and PDRS content; direct corrections and alternate approaches; consult on complex cases; establish precedent; and elevate unresolved technical or policy issues.

• Provide strategic and technical leadership for enterprise post-deployment governance, measurement, monitoring and reporting, and PDM and PDRS standards.
• Define risk-proportionate requirements across pilot, full implementation, post-deployment change, recurring PDRS, and legacy-product pathways.
• Establish standards for signals, metrics, formulas, baselines, targets, thresholds, uncertainty, evidence confidence, outcomes, and subgroup interpretation.
• Define monitoring-readiness expectations for sources, owners, collection methods, cadence, versions, limitations, lineage, Data Cards, Model Cards, handoffs, and sustainable ownership.
• Provide authoritative SME review of Governance Operations Product Lead assessment content and evidence for policy alignment, sufficiency, traceability, methodological adequacy, and decision readiness.
• Apply data science, statistical, AI/ML engineering, and systems methods to assess metric validity, source fitness, threshold logic, analyses, limitations, and conclusions.
• Review observability, logging, telemetry, workflow signals, version context, change detection, and monitoring and reporting continuity through significant changes.
• Own complex or precedent-setting questions involving monitoring, thresholds, evidence insufficiency, vendor limitations, significant change, revalidation continuity, lifecycle action, PDRS, or CAIO escalation.
• Recommend corrections, alternate methods, interim controls, additional evidence, action plans, re-review, retesting, or revalidation.
• Set precedent, issue final AVM direction, and elevate policy, clinical, cross-domain, or enterprise impasses.
• Lead PDRS templates and rubrics, evidence-confidence and escalation methods, metric libraries, executive presentation standards, and governance acceptance criteria.
• Convert recurring gaps into policy, playbooks, standard findings, rubrics, examples, training, calibration, and Product Lead enablement.
• Define enterprise requirements for TRex workflows, evidence objects, traceability, dashboards, portfolio visibility, and reusable governance capabilities.
• Coordinate with product teams, vendors, platforms, legal, committees, and enterprise groups on methods, tooling, specifications, and ownership.
• Provide clear complex-case findings that communicate limitations, confidence, required actions, and escalation triggers to technical and non-technical audiences.
• Mentor and calibrate engineers, analysts, and Product Leads; foster consistent methods and cross-lane coordination with Validation & Evaluation.
• Support audit sampling, quality assurance, enterprise learning, and continuous improvement while preserving AVM’s review-and-consultation boundary.
• Provide mentorship, guidance, and technical leadership to junior engineers. May have supervisory responsibilities.

Qualifications

• A master’s degree in engineering, computer science, mathematics, health science, or a related field with 7 years of relevant experience, or a bachelor’s degree with 9 years of relevant experience.
• Extensive (7+ years) experience applying AI and machine learning in production healthcare environments or similar highly regulated or technology focused industries, showcasing an acute understanding of healthcare technology.
• Demonstrated leadership in managing complex projects, with a proven ability to navigate intricate project requirements and deliver successful outcomes
• Proven success in fostering collaboration across diverse teams and effectively communicating complex technical concepts to non-technical stakeholders.
• Demonstrated expertise in cloud infrastructure environment and software development tools.
• Experience working with large, complex, and heterogeneous data sets, preferably in healthcare.
• Strong skills in AI/ML techniques and frameworks.
• Expertise with best practices in data engineering, data science, AI Engineering, and the MLOps communities.
• In-depth knowledge of healthcare domain, including clinical workflows, electronic health records, medical terminologies, regulatory requirements, and industry standards.
• Demonstrated leadership in administration, education, software development, and technical reporting.

• Experience mentoring and training less-experienced team members, coupled with strong interpersonal, communication, and time management skills.

  Preferred Qualifications:

• A Ph.D. or other doctorate is preferred.
• Experience with healthcare industry informatics standards, best practices, and common data models. Participation in national or international standards organizations or other domain-specific professional organizations, or extensive implementation experience with common data, development, and deployment standards.
• Excellent communication, collaboration, and stakeholder management skills, with the ability to effectively engage with diverse stakeholders and translate complex technical concepts and results to non-technical audiences.
• Demonstrated experience leading technical/quantitative teams in a regulated environment.
• Familiarity with systems or quality engineering best practices, regulatory standards, and compliance frameworks, with the ability to adapt these effectively to different project scenarios.
• Demonstrated experience creating risk management files and verification/validation strategies for digital health technology products within the healthcare industry.
• Demonstrated expertise in user-centered design, human factors engineering, usability testing methodologies, and evaluation across AI product development. Ability to lead expert reviews using established usability practices and methods. Presents findings in easy-to-understand terms for the business or clinical practice.
• Strong problem-solving abilities, critical thinking skills, and a passion for driving innovation and positive change in healthcare through AI technology.
• Demonstrated hands-on leadership using the TRex assessment application to govern AI tools deployed in EPIC, ANIMATE, and comparable clinical environments, including post-deployment standards, metric thresholds, evidence confidence, significant-change review, revalidation, and executive escalation.
  

Exemption Status

Exempt

Compensation Detail

$163,280.00 - $236,745.60/ year. Education, experience and tenure may be considered along with internal equity when job offers are extended.

Benefits Eligible

Yes

Schedule

Full Time

Hours/Pay Period

80

Schedule Details

M-F 8am-5pm This is a hybrid position and must be located within 100 miles of a Mayo Clinic campus for occasional on-site expectations based on business needs.

Weekend Schedule

NA

International Assignment

No

Site Description

Just as our reputation has spread beyond our Minnesota roots, so have our locations. Today, our employees are located at our three major campuses in Phoenix/Scottsdale, Arizona, Jacksonville, Florida, Rochester, Minnesota, and at Mayo Clinic Health System campuses throughout Midwestern communities, and at our international locations. Each Mayo Clinic location is a special place where our employees thrive in both their work and personal lives. Learn more about what each unique Mayo Clinic campus has to offer, and where your best fit is.

Equal Opportunity

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender identity, sexual orientation, national origin, protected veteran status or disability status. Learn more about the 'EOE is the Law'. Mayo Clinic participates in E-Verify and may provide the Social Security Administration and, if necessary, the Department of Homeland Security with information from each new employee's Form I-9 to confirm work authorization.