MD AI Evaluation Specialist (Part-Time)

Mindrift (Toloka); United States; Remote; USD 148,000 - 210,000; Part time

Job summary

Mindrift is seeking US-based physicians (MD/DO) to join an AI evaluation program evaluating clinical EHR vignettes. Projects are part-time, non-permanent and task-based, with adjudication by physician reviewers. You'll score diagnostic reasoning, rate vignette quality, and write concise rationales to help improve AI systems.

This opportunity suits active clinicians seeking flexible, project-based work. Join a dynamic platform that connects medical expertise with AI development, offering

Qualifications

• Medical degree (MD or DO) and active US medical license.
• Board certification or completed residency with recent direct patient care.
• Broad, multi-system diagnostic and treatment experience.
• Strong clinical reasoning and application of guidelines.
• Prior experience in medical AI evaluation or annotation is a plus.
• Strong written English (C1+).

Responsibilities

• Evaluate clinical EHR vignettes paired with questions and distractors.
• Score vignette accuracy, completeness, and trap quality.
• Check whether the reasoning chain reaches the answer from vignette facts.
• Write short rationales and participate in adjudication.

Skills

Strong written English (C1+)

Education

MD or DO

Active US medical license

Board certification or completed residency

Job description

Mindrift is seeking US-based physicians (MD/DO) to join an AI evaluation program evaluating clinical EHR vignettes. Projects are part-time, non-permanent and task-based, with adjudication by physician reviewers. You'll score diagnostic reasoning, rate vignette quality, and write concise rationales to help improve AI systems.

This opportunity suits active clinicians seeking flexible, project-based work. Join a dynamic platform that connects medical expertise with AI development, offering