Part-Time MD/DO Physician AI Evaluator

Mindrift; Philadelphia (Philadelphia County); Remote; USD 114,000 - 162,000; Part time

Job summary

Mindrift invites US-based physicians to participate in a part-time, project-based AI evaluation program. You will review clinical EHR vignettes, score reasoning quality, and provide concise rationales to improve AI systems.

Tasks are paid per accepted item, with rates up to $100 per hour depending on qualification tier and efficiency. The role emphasizes independent work, strict adherence to clinical guidelines, and clear written communication in English.

Qualifications

• MD or DO required with active US medical license
• Board certification or completed residency with recent direct patient care
• Broad diagnostic and treatment experience across internal, family, emergency or hospital medicine
• Strong clinical reasoning: differential diagnosis, management, guidelines
• Prior experience in medical AI evaluation or annotation is a plus
• Excellent written English (C1+)

Responsibilities

• Evaluate clinical EHR vignettes paired with questions, proposed answers and distractors
• Score reasoning quality across vignette accuracy, completeness and trap quality
• Assess whether reasoning traces reach the correct answer from vignette facts
• Write short rationales and participate in physician adjudication
• Work in three blind reads and contribute to project quality control

Skills

MD/DO

US medical license

Board certification

Residency training

Clinical reasoning

AI evaluation experience

Written English

Education

MD or DO degree

Job description

Mindrift invites US-based physicians to participate in a part-time, project-based AI evaluation program. You will review clinical EHR vignettes, score reasoning quality, and provide concise rationales to improve AI systems.

Tasks are paid per accepted item, with rates up to $100 per hour depending on qualification tier and efficiency. The role emphasizes independent work, strict adherence to clinical guidelines, and clear written communication in English.