Clinical AI Evaluation Specialist (MD/DO)

Weekday AI; United States; On-site; USD 170,000 - 243,000; Full time

Job summary

Weekday AI is seeking residency-trained physicians to join a shared expert pool for non-clinical work evaluating AI systems. You will apply clinical judgment to grading criteria, dialogue evaluation, and structured annotation that guides model measurement.

Roles are project-varied and may shift across streams based on specialty, availability, and interest; there is no live patient care or responsibility for diagnosis.

Qualifications

• MD or DO with completed residency in any specialty
• Active, unrestricted medical license in country of practice
• 2+ years post-residency clinical experience, practicing or previously practicing
• Comfort writing structured clinical rationale
• Written and spoken English fluency
• Minimum 20 hours per week, with concentrated hours when stream is time-boxed

Responsibilities

• Grading criteria development for model evaluation
• Clinical dialogue evaluation for accuracy, safety and completeness
• Clinical reasoning annotation detailing differential diagnoses
• Output review to flag hallucinations and unsafe advice
• Guideline authoring to define edge cases and standards of care
• Difficult-case writing to probe model reasoning limits

Skills

Clinical reasoning

Structured writing

English fluency

Clincial judgement

Education

MD or DO

Residency completed

Job description

Weekday AI is seeking residency-trained physicians to join a shared expert pool for non-clinical work evaluating AI systems. You will apply clinical judgment to grading criteria, dialogue evaluation, and structured annotation that guides model measurement.

Roles are project-varied and may shift across streams based on specialty, availability, and interest; there is no live patient care or responsibility for diagnosis.