Medical AI Evaluation Specialist (Remote)

YO AI Labs; United States; Remote; USD 60,000 - 100,000; Full time

Job summary

YO AI Labs is seeking Medical Evaluation Specialists (contractor) to contribute clinical expertise for evaluating and improving AI systems. You will create and validate high-difficulty medical Q&A pairs, test AI interpretations, and document reasoning with citations.

This remote role suits medical students, residents, physicians, or biomedical professionals who can work independently, meet deadlines, and produce precise, evidence-based content for AI benchmarking.

Qualifications

• Medical training and clinical expertise required.
• Ability to locate, interpret, and synthesize medical literature.
• Excellent attention to detail and precise written English.
• Capability to create clinically nuanced, high-quality questions.
• Experience in medical content development or evaluation is a plus.

Responsibilities

• Create high-difficulty medical question-and-answer pairs covering diagnosis, pathophysiology, pharmacology, guidelines, and decision-making.
• Develop questions that require synthesis and genuine medical reasoning.
• Research and verify answers using primary literature, guidelines, and reviews.
• Document rationale behind answers with appropriate citations.
• Evaluate AI-generated responses for clinical accuracy and evidence alignment.
• Refine questions to increase complexity while maintaining validity.
• Review for ambiguity, errors, and inconsistencies; ensure clarity and defensibility.
• Incorporate reviewer feedback and adapt to project guidelines.

Skills

Medical training

Research & Source Triangulation

Attention to Detail

Written Precision

Analytical Thinking

Clinical Reasoning

Medical Literature Review

Evidence Synthesis

Self-Direction & Reliability

Medical Question Development

Education

Medical degree or equivalent clinical training

Current or recent medical student, resident, or physician

Job description

YO AI Labs is seeking Medical Evaluation Specialists (contractor) to contribute clinical expertise for evaluating and improving AI systems. You will create and validate high-difficulty medical Q&A pairs, test AI interpretations, and document reasoning with citations.

This remote role suits medical students, residents, physicians, or biomedical professionals who can work independently, meet deadlines, and produce precise, evidence-based content for AI benchmarking.