Medical Expert AI Evaluation Specialist

Weekday 1; Canada; Remote; CAD 179,000 - 248,000; Part time

Benefits offered by this job

Fully remote

Asynchronous work

Weekly payments

Flexible schedule

Job summary

Weekday 1 is seeking experienced medical professionals to join a fully remote, asynchronous engagement focused on training and evaluating AI models for healthcare. You will develop realistic clinical scenarios, generate reference-quality solutions, and assess AI performance using evidence-based guidelines, with weekly payments and a flexible schedule.

This role is suitable for attending physicians, residents in the final year, and fellows across multiple specialties, including family medicine,

Qualifications

• Board-certified physician with a current, active, and unrestricted medical license.
• Final year of accredited residency program, or board-eligible for recent grads.
• Board-certified or board-eligible in primary specialty.
• Physicians from a wide range of specialties are encouraged to apply.

Responsibilities

• Develop Clinical Scenarios for specialty-specific cases reflecting real-world practice.
• Create reference-quality clinical responses aligned with guidelines.
• Evaluate AI performance using structured evaluation frameworks and rubrics.
• Collaborate with clinical experts in onboarding and calibration meetings.

Skills

Clinical evaluation

Medical writing

Education/research

Education

Medical degree (MD)

Board-certified physician

Active medical license

Residency completion

Job description

This role is for one of our clients

Compensation: $130-$180 per hour

Location requirements

Canada, United States

Join a leading AI research initiative focused on advancing healthcare-focused artificial intelligence. We are seeking experienced medical professionals to contribute their clinical expertise toward training and evaluating next-generation AI models capable of sophisticated medical reasoning.

In this role, you will help create realistic clinical scenarios, evaluate AI-generated responses using evidence-based medical standards, and provide expert insights that improve how AI systems reason about diagnosis, treatment, and patient care.

This opportunity is ideal for physicians across a broad range of medical specialties who are passionate about applying their expertise to the future of healthcare technology.

Requirements

Key Responsibilities

Develop Clinical Scenarios

• Create realistic, specialty-specific clinical cases that reflect real-world medical practice.
• Design prompts covering areas such as:
• Diagnostic reasoning
• Treatment planning
• Risk stratification
• Differential diagnosis
• Guideline-based clinical decision-making

Create Reference-Quality Solutions

• Produce high-quality clinical responses that reflect attending physician-level reasoning.
• Ensure recommendations align with current evidence-based practices and accepted clinical guidelines.

Evaluate AI Performance

• Assess AI-generated medical responses using structured evaluation frameworks and scoring rubrics.
• Identify strengths, weaknesses, omissions, and reasoning errors.
• Provide detailed written feedback to improve model performance and clinical accuracy.

Collaborate with Clinical Experts

• Participate in onboarding sessions and specialty calibration meetings.
• Work with multidisciplinary research teams to maintain consistency and quality across medical evaluations.

Required Qualifications

Applicants should meet one of the following criteria:

Attending Physicians

• Board-certified physician with a current, active, and unrestricted medical license.

Resident Physicians

• Currently in the final year of an accredited residency program.
• Recent residency graduates must be board-eligible.

Fellows

• Board-certified or board-eligible in their primary specialty.
• Current, active, and unrestricted medical license.

Physicians from a wide range of specialties are encouraged to apply, including but not limited to:

• Family Medicine
• Pediatrics
• Anesthesiology
• General Surgery
• ENT (Otolaryngology)
• Physical Medicine & Rehabilitation (PM&R)
• Occupational Medicine
• Other medical and surgical specialties

Preferred Qualifications

• Strong understanding of evidence-based medicine and clinical practice guidelines.
• Excellent written communication skills.
• Experience with medical education, clinical research, peer review, or healthcare quality initiatives.
• Interest in artificial intelligence and healthcare innovation.

Engagement Details

• Fully remote with 100% asynchronous work.
• Typical commitment of approximately 20 hours per week, with opportunities for additional hours based on project needs.
• Flexible schedule.
• Weekly payments.
• Rolling onboarding with ongoing project opportunities.
• Engagement duration is reviewed periodically based on project requirements and performance.

Why Join

• Help shape the future of AI-powered healthcare.
• Apply your clinical expertise to improve advanced medical reasoning systems.
• Collaborate with leading AI researchers and healthcare professionals.
• Contribute to high-impact projects that advance patient-centered artificial intelligence.

Equal Opportunity Statement

All qualified applicants will be considered without regard to legally protected characteristics. Reasonable accommodations are available upon request.

Contract Information

• Independent contractor engagement.
• Fully remote work completed on your own schedule.
• Project duration may be extended, modified, or concluded based on business needs and individual performance.
• Work does not involve access to confidential or proprietary information from any employer, client, or institution.
• Weekly payments are processed through supported payment platforms based on approved work completed.
• Please note that visa sponsorship is not available for this opportunity.