Remote MD/DO Medical AI Evaluation Specialist

Toloka; Indiana; Remote; USD 41,000 - 83,000; Part time

Benefits offered by this job

Remote work

Onboarding reward

Flexible schedule

Job summary

Mindrift seeks MD/DO licensed US physicians to evaluate clinical EHR vignettes for an AI evaluation project. You will review written cases, apply acceptance criteria, and provide rationales.

Three independent reads followed by adjudication ensure quality and consistency, with a focus on multi-system diagnosis and treatment considerations. Contributors work remotely with flexible scheduling, allowing you to fit the project around other commitments.

Qualifications

• MD/DO with active, unrestricted US license.
• Board certification or residency with direct patient care experience.
• Broad multi-system diagnostic and treatment experience.
• Strong clinical reasoning and evidence-based guidelines.
• Prior experience in medical AI evaluation or medical QA is a plus.
• Strong written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes with questions, proposed answers, and distractors across multiple systems.
• Score clinical reasoning quality, vignette accuracy, and trap quality.
• Check whether reasoning traces lead from vignette facts to the answer and write concise rationales.

Skills

Active US medical license

MD or DO degree

Board certification or residency

Clinical reasoning

Medical AI evaluation experience

Strong written English

Education

MD or DO degree

Board certification or residency completed

Job description

Mindrift seeks MD/DO licensed US physicians to evaluate clinical EHR vignettes for an AI evaluation project. You will review written cases, apply acceptance criteria, and provide rationales.

Three independent reads followed by adjudication ensure quality and consistency, with a focus on multi-system diagnosis and treatment considerations. Contributors work remotely with flexible scheduling, allowing you to fit the project around other commitments.