Remote Medical Evaluation Specialist for AI Testing

YO AI Labs; Chicago (IL); Remote; USD 55,000 - 124,000; Part time

Job summary

YO AI Labs seeks Medical Evaluation Specialists (medical students, residents, physicians, and biomedical professionals) to create and validate high-difficulty medical questions and answers for testing AI systems. You will craft items that challenge clinical reasoning and synthesize complex medical evidence.

Responsibilities include researching primary literature, documenting rationales with citations, reviewing for accuracy, and refining questions to maintain clinical validity within evolving

Qualifications

• Medical training or clinical expertise required.
• Strong writing and documentation skills in English.
• Ability to locate, interpret, and synthesize medical literature.
• Experience writing medical questions or content is a plus.

Responsibilities

• Create original, high-difficulty medical question-and-answer pairs covering diagnosis, pathophysiology, pharmacology, clinical guidelines, and clinical decision-making.
• Develop questions that require synthesis, interpretation, and genuine medical reasoning rather than simple recall.
• Research and verify answers using primary literature, clinical guidelines, systematic reviews, and authoritative medical references.
• Document the rationale behind answers and provide supporting citations.
• Evaluate AI-generated responses for clinical accuracy, completeness, reasoning quality, and adherence to evidence.

Skills

Medical Training

Research & Source Triangulation

Attention to Detail

Written Precision

Analytical Thinking

Clinical Reasoning

Medical Literature Review

Evidence Synthesis

Self-Direction & Reliability

Medical Question Development

Job description

YO AI Labs seeks Medical Evaluation Specialists (medical students, residents, physicians, and biomedical professionals) to create and validate high-difficulty medical questions and answers for testing AI systems. You will craft items that challenge clinical reasoning and synthesize complex medical evidence.

Responsibilities include researching primary literature, documenting rationales with citations, reviewing for accuracy, and refining questions to maintain clinical validity within evolving