Part-Time Medical AI Evaluation Specialist

24-Mag Llc; New York (NY); Remote; USD 96,000 - 165,000; Part time

Job summary

24-MAG LLC offers a part-time remote engagement for licensed physicians actively practicing medicine. You will contribute to advanced medical AI evaluation through clinical reasoning, scenario design, and benchmark development.

Opportunities span across specialties with flexible scheduling. The project runs for about one month, with potential extension based on performance. No compensation details are provided in the source materials, and you will work remotely to support research teams.

Qualifications

• Licensed physician currently engaged in active clinical practice.
• MD, DO, or equivalent medical qualification.
• Open to physicians across medical specialties.
• Backgrounds may include Public Health, Pediatrics, Emergency Medicine, Obstetrics & Gynecology, or Gynecological Oncology.
• Strong clinical decision-making experience.
• Solid understanding of evidence-based medicine.
• Ability to assess complex medical reasoning objectively.
• Strong analytical and critical-thinking capabilities.
• Excellent written and verbal communication skills.
• Comfortable explaining nuanced clinical decisions clearly.
• Able to identify gaps in knowledge or reasoning.
• Interest in AI applications to clinical practice.
• Comfortable collaborating in remote environment.
• Able to work independently around existing clinical commitments.

Responsibilities

• Evaluate AI performance on realistic medical reasoning tasks.
• Assess whether conclusions are supported.
• Identify gaps in medical knowledge or decision-making.
• Review model reasoning against evidence-based practice.
• Distinguish clinically significant errors from minor limitations.
• Create realistic medical scenarios for model evaluation.
• Design cases requiring nuanced clinical judgement.
• Develop evaluation frameworks for medical AI systems.
• Provide structured rationale for evaluation decisions.
• Collaborate with research teams and give actionable feedback.
• Support development of clinically meaningful benchmarks.

Skills

Licensed physician

MD/DO

Clinical reasoning

Medical knowledge

Remote collaboration

Education

MD/DO degree

Job description

24-MAG LLC offers a part-time remote engagement for licensed physicians actively practicing medicine. You will contribute to advanced medical AI evaluation through clinical reasoning, scenario design, and benchmark development.

Opportunities span across specialties with flexible scheduling. The project runs for about one month, with potential extension based on performance. No compensation details are provided in the source materials, and you will work remotely to support research teams.