Remote Healthcare AI Quality Evaluator

Medix™; United States; Remote; USD 103,000 - 172,000; Full time

Benefits offered by this job

Fully remote

Flexible schedule

Job summary

Medix is seeking experienced healthcare professionals to support a major AI initiative focused on improving healthcare AI models. This is a fully remote, 1099 contract role with flexible scheduling (5–40 hours per week) and an hourly pay range of $40 to $125.

Ideal candidates have 5+ years of hands-on healthcare experience, active clinical licensure, and strong analytical and written communication skills. Prior AI or data annotation exposure is a plus; you will evaluate AI outputs and develop

Qualifications

• 5+ years of relevant healthcare experience
• Bachelor's, Master's, or PhD preferred; equivalent advanced practical experience may be considered
• Current or recent clinical license for clinical-track candidates
• Strong analytical and written communication skills
• Ability to interpret healthcare documentation and apply professional judgment
• Exceptional attention to detail and ability to work independently
• Availability to work 5–40 hours per week
• Reliable computer and high-speed internet connection
• Must reside in the United States
• Prior AI, LLM testing, data annotation, or model evaluation experience is a plus

Responsibilities

• Evaluate AI model responses to healthcare-related prompts and scenarios.
• Assess accuracy, safety, clinical reasoning, documentation, and workflow logic.
• Create and apply detailed grading rubrics to measure AI performance.
• Validate healthcare terminology, documentation requirements, compliance standards, and operational workflows.
• Identify errors, inconsistencies, omissions, and potentially unsafe recommendations.
• Work independently using provided AI tools, project instructions, and reference materials.
• Apply current guidelines and real-world healthcare experience when evaluating AI outputs.

Skills

Healthcare experience

Analytical skills

Written communication

Education

Higher degree preferred

Job description

Medix is seeking experienced healthcare professionals to support a major AI initiative focused on improving healthcare AI models. This is a fully remote, 1099 contract role with flexible scheduling (5–40 hours per week) and an hourly pay range of $40 to $125.

Ideal candidates have 5+ years of hands-on healthcare experience, active clinical licensure, and strong analytical and written communication skills. Prior AI or data annotation exposure is a plus; you will evaluate AI outputs and develop