Physician AI Evaluator

AI Trainer Jobs; United States; Remote; USD 83,000 - 124,000; Full time

Job summary

AuraOne is seeking a Physician AI Evaluator to review AI outputs touching physician practice. You will assess clinical reasoning, dosing logic, and guideline adherence, flag safety issues, and document corrected reasoning for model training.

Active medical license and hands-on case review experience are required. Strong written reasoning and ability to apply rubrics consistently across batches are essential. Remote work with flexible hours.

Qualifications

• Active or recent license/board certification in physician or adjacent specialty.
• Hands-on clinical experience reading patient cases and writing assessments.
• Ability to apply multi-page rubrics consistently across large batches.
• Clear written reasoning citing guidelines, evidence, or standard of care.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Review AI outputs against physician guidelines and standard of care for Physician AI Evaluator assignments.
• Grade differential reasoning, dosing, and red-flag handling on a structured rubric.
• Flag patient-safety issues with severity tags that match clinical-incident taxonomies.
• Capture the corrected clinical reasoning so the modeling team can train on it.
• Adjudicate disputed cases against published evidence or specialty guidelines.

Skills

Active medical license

Clinical case review

Rubric application

Written reasoning

Weekly availability

Education

Medical license/board certification

Tools

Guideline databases

Structured assessment tools

Job description

About the role

Physician AI Evaluator is a remote clinical-review track for evaluating AI outputs that touch physician. Reviewers grade differential reasoning, dosing logic, and guideline adherence; flag patient-safety issues; and document the corrected clinical reasoning so the modeling team can close the gap.
Physician AI is only useful if a credentialed clinician would sign their name to the output. AuraOne uses licensed and credentialed reviewers to set the bar for clinical reasoning, dosing, and guideline adherence in regulated workflows.
Review clinical and biomedical AI outputs. Your safety judgment is the point.

Responsibilities

• Review AI outputs against current physician clinical guidelines and standard of care for Physician AI Evaluator assignments.
• Grade differential reasoning, dosing, and red-flag handling on a structured rubric.
• Flag patient-safety issues with severity tags that match clinical-incident taxonomies.
• Capture the corrected clinical reasoning so the modeling team can train on it.
• Adjudicate disputed cases against published evidence or specialty guidelines.

What you should bring

• Active or recent license / board certification in physician or an adjacent specialty for Physician AI Evaluator work.
• Hands-on clinical experience reading patient cases and writing structured assessments.
• Comfort applying multi-page rubrics consistently across long batches.
• Clear written reasoning that cites guidelines, evidence, or standard of care.
• Reliable async availability for at least 10 hours per week.

Role signals

Example tasks

• Grade a model's differential-diagnosis reasoning on a physician case and write a 2-3 sentence assessment.
• Flag a dosing miscalculation with the right severity tag and corrected dose.
• Adjudicate a disputed case between two reviewers using current specialty guidelines.
• Audit a 25-case batch for rubric consistency and report drift to the program lead.

Useful experience

• Prior work as a guideline reviewer, peer reviewer, or clinical educator.
• Familiarity with AI-assisted clinical decision support and its failure modes.
• Bilingual clinical experience for non-English patient cases.

Compensation and schedule

Hourly rate confirmed after the interview process.
Expected arrangement: contractor , with program-defined task volume and review pacing. Placement depends on current program demand and reviewer confirmation.

Skills used in matching

• Clinical reasoning
• Guideline review
• Patient-safety assessment
• Structured clinical writing
• Physician
• Medicine and healthcare
• AI evaluation
• Rubric writing
• Expert review

Application boundary

Creating a specialist profile records your experience and preferences. Starting role intake is a separate action that attaches this role to your candidate record.

Specialist intake

The intake preserves your chosen role, the visible terms, and source attribution for reviewer context.

• 01 Confirm profile and eligibility details.
• 02 Attach this role deliberately.
• 03 Receive a human review decision or follow-up.
  Placement timing depends on program demand and reviewer confirmation.