Physician AI QA Evaluator (MD/DO) Part-Time

Mindrift; Fort Collins (CO); Remote; USD 83,000 - 138,000; Part time

Benefits offered by this job

Part-time projects

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies. We are seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes for an AI evaluation program focused on testing, evaluating, and improving AI systems.

Contributors may work 10–20 hours per week on a part-time, non-permanent basis, with compensation per accepted task up to $100/hour. Ideal candidates have MD/DO, active US licensure, and strong English writing skills.

Qualifications

• MD or DO with active US medical license.
• Board certification or completed residency with recent direct patient care.
• Broad, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine.
• Strong clinical reasoning: differential diagnosis and management.
• Experience in medical AI evaluation, content review, or QA is a plus.
• Excellent written English skills (C1+).

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers, and distractor options across multiple body systems.
• Score clinical reasoning quality, vignette accuracy, completeness, and trap quality.
• Assess whether the reasoning chain reaches the correct answer from vignette facts.
• Write concise rationales and participate in adjudication rounds.

Skills

US medical license

Board certification / residency

Clinical reasoning

Medical AI experience

Strong written English

Education

MD or DO

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies. We are seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes for an AI evaluation program focused on testing, evaluating, and improving AI systems.

Contributors may work 10–20 hours per week on a part-time, non-permanent basis, with compensation per accepted task up to $100/hour. Ideal candidates have MD/DO, active US licensure, and strong English writing skills.