Healthcare AI Systems Evaluator

Empathy

United States

On-site

USD 82,656 - 165,312

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

mpathic.ai is seeking experienced healthcare systems professionals to evaluate and improve AI models that support clinical operations, administrative workflows, and healthcare technology. You will review real-world scenarios, evaluate AI-generated responses, and provide expert feedback to improve reliability for providers, staff, and patients.

Bring deep knowledge of healthcare workflows across hospitals and health systems, familiarity with EHR platforms, and the ability to communicate complex

Qualifications

  • 5+ years in healthcare operations or related healthcare systems role.
  • Strong understanding of hospital and health system workflows.
  • Experience with EHR platforms like Epic, Cerner/Oracle Health, Meditech, athenahealth.
  • Familiarity with HL7/FHIR CCD/C-CDA data exchange.
  • Knowledge of HIPAA and CMS requirements.
  • Strong written English and clear communication.

Responsibilities

  • Evaluate AI-generated responses for accuracy and alignment with real-world operations.
  • Review scenarios involving electronic health records (EHRs), clinical documentation, care coordination, scheduling, referrals, revenue cycle, and patient access workflows.
  • Assess AI outputs for consistency with healthcare regulations, industry standards, and operational best practices.
  • Apply structured evaluation guidelines to annotate model responses and identify errors, omissions, and opportunities for improvement.
  • Provide clear, actionable feedback that helps improve AI performance across complex healthcare workflows.
  • Evaluate AI handling of interoperability, health information exchange, and healthcare data management concepts.

Skills

Healthcare ops
EHR workflows
Regulatory knowledge
English writing

Tools

Epic
Cerner
Meditech
athenahealth

Job description

mpathic.ai is seeking experienced healthcare systems professionals to evaluate and improve AI models that support clinical operations, administrative workflows, and healthcare technology. You will review real-world scenarios, evaluate AI-generated responses, and provide expert feedback to improve reliability for providers, staff, and patients.

Bring deep knowledge of healthcare workflows across hospitals and health systems, familiarity with EHR platforms, and the ability to communicate complex

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI-Savvy Healthcare Systems Evaluator for Clinical Ops
AI-Savvy Healthcare Systems Evaluator for Clinical Ops

mpathic • United States

Remote
USD 110,000 - 193,000
Clinical AI Evaluation Specialist
Clinical AI Evaluation Specialist

mpathic • United States

Remote
USD 83,000 - 165,000
Healthcare Systems Specialist (Contract)
Healthcare Systems Specialist (Contract)

Mpathic • United States

On-site
USD 83,000 - 152,000
Clinical AI Quality Evaluator - Physician
Clinical AI Quality Evaluator - Physician

Mpathic • United States

On-site
USD 110,000 - 220,000
Healthcare Systems Specialist (Contract)
Healthcare Systems Specialist (Contract)

mpathic • United States

Remote
USD 110,000 - 193,000
Healthcare AI Evaluation Specialist — Independent Technical Reviewer
Healthcare AI Evaluation Specialist — Independent Technical Reviewer

Mpathic • Bellevue (WA)

On-site
USD 83,000 - 165,000
Clinical AI Safety Expert (Contract)
Clinical AI Safety Expert (Contract)

mpathic • United States

Remote
USD 138,000 - 276,000
AI/ML Evaluation Reviewer- Healthcare AI (Contract)
AI/ML Evaluation Reviewer- Healthcare AI (Contract)

Mpathic • Utah

Hybrid
USD 83,000 - 124,000
AI/ML Evaluation Reviewer- Healthcare AI (Contract)
AI/ML Evaluation Reviewer- Healthcare AI (Contract)

mpathic • United States

Remote
USD 83,000 - 165,000
AI/ML Evaluation Reviewer- Healthcare AI (Contract)
AI/ML Evaluation Reviewer- Healthcare AI (Contract)

Mpathic • Bellevue (WA)

On-site
USD 83,000 - 165,000