Healthcare SME: Rubric Design & AI Evaluation (Remote)

SuperAnnotate AI, Inc.

United States

Remote

USD 83,000 - 138,000

Part time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

SME Careers, a fast-growing AI Data Services company and subsidiary of SuperAnnotate, seeks Healthcare Subject Matter Expert contractors in the United States. You will create expert healthcare questions, write detailed scoring rubrics, and critically evaluate AI-generated clinical content.

This remote hourly role requires 3+ years of clinical experience, strong English, and a proven ability to produce consistent, high-quality written work across RLHF/SFT workflows.

Qualifications

  • Professional degree, license, or recognized qualification in a healthcare discipline.
  • Strong English proficiency (C1+).
  • 3+ years of professional or clinical experience; senior level preferred.
  • Expertise in a clearly defined specialty or subspecialty.

Responsibilities

  • Create expert healthcare questions exposing AI system limitations.
  • Design detailed rubrics defining required clinical reasoning and caveats.
  • Evaluate AI outputs for accuracy, safety, and completeness.
  • Critique peer rubrics for ambiguity and missing criteria.
  • Develop reference answers and gold-standard content.
  • Support AI training with questions, rubrics, and evaluations.

Skills

Healthcare
Medicine
Clinical Reasoning
AI Evaluation
Rubric Design
Medical Education
English Proficiency
Expert Review

Education

Medical degree
Nursing license
Pharmacy degree

Job description

SME Careers, a fast-growing AI Data Services company and subsidiary of SuperAnnotate, seeks Healthcare Subject Matter Expert contractors in the United States. You will create expert healthcare questions, write detailed scoring rubrics, and critically evaluate AI-generated clinical content.

This remote hourly role requires 3+ years of clinical experience, strong English, and a proven ability to produce consistent, high-quality written work across RLHF/SFT workflows.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Healthcare Rubric Designer & AI Evaluator
Remote Healthcare Rubric Designer & AI Evaluator

Joinkeywork • United States

Remote
USD 96,000 - 165,000
Healthcare Specialist – Rubric Design
Healthcare Specialist – Rubric Design

SME Careers • United States

Remote
USD 96,000 - 165,000
Healthcare Specialist – Rubric Design
Healthcare Specialist – Rubric Design

SuperAnnotate AI, Inc. • United States

Remote
USD 83,000 - 138,000
Remote Healthcare AI Rubric & Evaluation Specialist
Remote Healthcare AI Rubric & Evaluation Specialist

OpenTrain AI, Inc. • United States

Hybrid
USD 114,000 - 162,000
Healthcare AI Rubric Evaluation Specialist
Healthcare AI Rubric Evaluation Specialist

OpenTrain AI, Inc. • United States

Hybrid
USD 114,000 - 162,000
Medical General Expert
Medical General Expert

SuperAnnotate AI, Inc. • United States

Remote
USD 102,000 - 146,000
258883 - Healthcare Subject Matter Expert (SME) – Medical AI
258883 - Healthcare Subject Matter Expert (SME) – Medical AI

Medix Staffing Solutions, Inc. • United States

Remote
USD 55,000 - 90,000
Remote Healthcare AI Evaluator – Clinical Expert
Remote Healthcare AI Evaluator – Clinical Expert

Mercor • San Francisco (CA)

Hybrid
GBP 82,000 - 123,000
Medical AI Content & Reasoning Reviewer (Remote)
Medical AI Content & Reasoning Reviewer (Remote)

SuperAnnotate AI, Inc. • United States

Remote
USD 102,000 - 146,000
Medical Doctor (Specialist)
Medical Doctor (Specialist)

SuperAnnotate AI, Inc. • United States

Remote
USD 124,000 - 207,000