Part time to Full time Medical AI Auditor - 260261

Medix™; United States; Remote; USD 55,000 - 172,000; Part time

Job summary

Medix Staffing Solutions, Inc. is seeking experienced healthcare clinicians to train and evaluate frontier AI models in a remote contract role. If you have independent clinical judgment and enjoy shaping AI safety and reliability, join us from anywhere in eligible regions.

You will design realistic clinical scenarios, run them through AI systems, and critically grade outputs to identify strengths and gaps, with flexible hours from 10 per week up to full-time commitments.

Qualifications

• Active clinical license in MD/DO/NP/PA/PharmD/RN or allied health with independent practice.
• Minimum 5 years of hands-on, direct clinical experience.
• Exceptional written English with ability to articulate complex medical reasoning clearly.
• Familiarity with AI/LLM tools and ability to differentiate plausible text from medically sound conclusions.

Responsibilities

• Design authentic, de-identified patient scenarios and prompts.
• Evaluate AI outputs against professional standards and identify concrete clinical failures.
• Compare outputs from identical prompts across AI models and justify the best choice.
• Develop detailed, evidence-based evaluation rubrics for correct clinical responses.
• Review and refine clinical test cases with peers across specialties.

Skills

Licensure

Clinical experience

English communication

AI/LLM tools familiarity

Analytical thinking

Job description

Clinical AI Trainer & Medical Domain Expert (Remote Contract)

Location:

100% Remote (Open to candidates residing in the United States, Canada, or the United Kingdom)

Compensation:

$40.00 – $125.00+ / hour USD (Project-dependent based on specialty, credential level, and task complexity)

Employment Type:

Flexible Independent Contract | Part-time to Full-time hours available (Minimum 10 hours/week recommended)

Position Overview

We are seeking experienced healthcare clinicians to help train and evaluate the next generation of frontier Artificial Intelligence models.

Please note:

This is not a direct patient care or clinical hospital position. Instead, you will apply your clinical judgment, real-world case experience, and diagnostic acumen to push AI models to their limits, evaluate complex clinical reasoning, and ensure AI systems are accurate, safe, and aligned with evidence-based standards.

In this role, you will design realistic clinical scenarios (e.g., complex SOAP notes, medication reconciliations, triage decisions, discharge summaries, or clinical protocols), run them through AI models, and critically grade the outputs to identify where the systems succeed or fail.

Key Responsibilities

• Design Realistic Clinical Scenarios: Author authentic, de-identified patient scenarios, prompts, and supporting clinical materials (labs, imaging reports, histories, medication lists) drawn from standard day-to-day practice.
• Model Evaluation & Failure Identification: Review AI-generated clinical plans, summaries, and recommendations against strict professional standards. Identify concrete clinical failures, such as missed drug interactions, unsupported diagnoses, diagnostic anchoring, omitted red flags, or fabricated/hallucinated data.
• Comparative Output Analysis: Compare competing AI model outputs generated from identical clinical prompts and data, select the superior response, and provide clear written rationales for your choice.
• Develop Grading Rubrics: Write detailed, evidence-based evaluation rubrics establishing what a correct clinical response must contain and explaining why an output passes or fails each criterion.
• Collaborative Peer Review: Review, refine, and provide feedback on clinical test cases created by other medical domain contributors across various specialties.

Required Qualifications

• Active or Recent Clinical Licensure: Licensed MD, DO, NP, PA, PharmD, RN, or allied health professional with independent clinical judgment (e.g., Physical Therapy, Respiratory Therapy, Midwifery). (Active license verification may be requested).
• Clinical Experience: 5+ years of hands‑on, direct clinical experience preferred (practical, hands‑on individual contributors with fewer years will also be considered).
• Communication & Reasoning: Exceptional written English skills with the ability to articulate complex medical reasoning clearly, explaining why an AI answer is clinically unsound rather than simply stating that it is incorrect.
• Technology Proficiency:
• Familiarity with general AI/LLM tools (e.g., ChatGPT, Claude) and the ability to differentiate between plausible‑sounding text and medically sound conclusions.
• Analytical Thinking: Strong attention to detail, comfort working independently with ambiguous prompts, and the discipline to fact‑check statements against underlying medical evidence.

Preferred Specialties

Clinical experience in one or more of the following areas is highly valued:

• Primary Care / Family Medicine / Internal Medicine
• Emergency Medicine / Critical Care / Acute Care
• Cardiology, Oncology, Neurology, or Specialty Pharmacy
• Pediatrics, Obstetrics, Behavioral Health, Geriatrics, or Perioperative Care

Why Join This Project?

• Total Schedule Flexibility: Work on your own schedule from anywhere within eligible locations; tasks allow for flexible multi‑day pacing.
• Direct Impact on Healthcare Tech: Help shape the safety, ethics, and reliability of clinical AI tools before they reach clinicians and patients.
• High‑Value Compensation: Competitive hourly project rates with no weekly cap on hours for strong performers.
• Model Access Included: Necessary platform and model tool access are fully provided.

Application & Onboarding Process

• Initial Screening: Review of your clinical background and licensure.
• Clinical Skills Assessment: Complete a self‑paced, domain‑specific evaluation testing your ability to review AI clinical outputs and identify nuanced medical errors (all assessment work must be your own original thought without generative AI tools).
• Project Onboarding: Upon successful assessment completion, you will receive orientation materials and gain access to paid clinical evaluation projects.