AI Medical Evaluator (MD/DO) — Part-Time, Project-Based

Mindrift (Toloka); United States; Remote; USD 58,000 - 96,000; Part time

Job summary

Mindrift is seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program. Projects are part-time and non-permanent, with compensation paid per accepted task.

Contributors will review vignette content across multiple medical domains, score reasoning quality, verify tracing accuracy, and provide concise rationales. The role emphasizes strong English writing and current US medical licensure.

Qualifications

• MD or DO with active US medical license.
• Board certification or recent residency with direct patient care.
• Broad, multi-system diagnostic and treatment experience.
• Strong clinical reasoning and guideline application.
• Experience in medical AI evaluation or annotation is a plus.
• Excellent written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes with questions and distractors across organ systems.
• Score the clinical reasoning quality and vignette accuracy.
• Check whether reasoning traces lead to the answer and write rationales.
• Work within three independent blind reads, followed by adjudication.

Skills

Medical license

Board certification

Clinical reasoning

English proficiency

Education

MD or DO degree

Job description

Mindrift is seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program. Projects are part-time and non-permanent, with compensation paid per accepted task.

Contributors will review vignette content across multiple medical domains, score reasoning quality, verify tracing accuracy, and provide concise rationales. The role emphasizes strong English writing and current US medical licensure.