Physician: Part-Time Medical AI Evaluator

Mindrift; United States; Remote; USD 107,000 - 179,000; Full time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based and flexible to fit busy schedules.

We are looking for US-based physicians (MD/DO) with active licenses to evaluate clinical EHR vignettes, covering multiple medical domains. Candidates will score reasoning quality, verify evidence, and write brief rationales, with adjudication and quality checks throughout.

Qualifications

• MD or DO with active, unrestricted US medical license.
• Board certification or completed residency training; direct patient care experience preferred.
• Broad, multi-system diagnostic and treatment experience.
• Strong clinical reasoning and knowledge of evidence-based guidelines.
• Experience in medical AI evaluation or medical annotation is a plus.
• Excellent written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes paired with questions and distractors.
• Score the clinical reasoning quality of benchmark items.
• Check whether reasoning traces lead to the answer using vignette facts.
• Write short rationales and participate in physician adjudication.

Skills

Strong clinical reasoning

Strong written English

AI evaluation experience

Education

MD or DO with active US medical license

Board certification or residency training

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based and flexible to fit busy schedules.

We are looking for US-based physicians (MD/DO) with active licenses to evaluate clinical EHR vignettes, covering multiple medical domains. Candidates will score reasoning quality, verify evidence, and write brief rationales, with adjudication and quality checks throughout.