Remote Medical Evaluation Specialist (Contract)

YO AI Labs; New York (NY); Remote; USD 45,000 - 89,000; Part time

Job summary

YO AI Labs seeks Medical Evaluation Specialists to craft and validate challenging medical questions for AI evaluation. Remote collaboration with physicians, students, and biomedical professionals to test AI systems on complex clinical reasoning and evidence interpretation.

You will research primary literature and guidelines, document rationale with citations, and refine questions for clarity and defensibility while adapting to evolving project standards.

Qualifications

• Proven medical training and ability to develop complex, high-difficulty questions.
• Strong research and source triangulation skills with primary literature.
• Exceptional attention to detail and written precision in English.
• Ability to synthesize evidence and apply clinical reasoning to cases.
• Experience in medical question development or clinical education is a plus.
• Familiarity with AI/ML concepts and medical AI applications is advantageous.

Responsibilities

• Create original high-difficulty medical Q&A pairs across diagnosis, pathophysiology, pharmacology, guidelines and decision-making.
• Develop questions requiring synthesis, interpretation and medical reasoning, not recall.
• Research and verify answers using primary literature, guidelines and reviews.
• Document rationale with supporting citations.
• Evaluate AI-generated responses for accuracy, completeness and evidence adherence.
• Identify overly simple questions and refine for greater complexity yet remain valid.
• Review content for ambiguity, errors, and inconsistencies; ensure clarity and defensibility.
• Incorporate reviewer feedback and adapt to evolving project guidelines.
• Contribute insights to establish rigorous benchmarks for medical AI evaluation.

Skills

Medical Training

Research & Source Triangulation

Attention to Detail

Written Precision

Analytical Thinking

Clinical Reasoning

Medical Literature Review

Evidence Synthesis

Self-Direction & Reliability

Medical Question Development

AI/ML familiarity

Job description

YO AI Labs seeks Medical Evaluation Specialists to craft and validate challenging medical questions for AI evaluation. Remote collaboration with physicians, students, and biomedical professionals to test AI systems on complex clinical reasoning and evidence interpretation.

You will research primary literature and guidelines, document rationale with citations, and refine questions for clarity and defensibility while adapting to evolving project standards.