Remote Healthcare AI Evaluation Specialist

Medix™; United States; Remote; USD 83,000 - 138,000; Part time

Benefits offered by this job

Remote work

Flexible schedule

Job summary

Medix seeks an AI Trainer & Evaluation Specialist – Healthcare for a 12-week contract, remote and flexible. You will design realistic clinical tasks and evaluate frontier AI models with your clinical judgment, focusing on accuracy, safety, and appropriate documentation.

You will build rubrics, flag potential failures, and compare model outputs to ensure professional standards are met, all while operating in a self-directed, long-form analytical role.

Qualifications

• Active clinical credential (MD/DO/NP/PA/PharmD/RN) and hands-on patient care experience.
• Exceptional written communication and ability to craft structured feedback.
• Comfortable working remotely with self-direction and independence.

Responsibilities

• Design authentic clinical tasks and supporting files for AI evaluation.
• Evaluate AI model outputs against professional standards and rubrics.
• Compare outputs from different AI models and document deficiencies.
• Develop granular rubrics detailing required content and safety checks.
• Identify and document clinical errors, missed interactions, or unsafe recommendations.
• Peer-review task sets and refine prompts across medical specialties.

Skills

Clinical expertise

Medical reasoning

Technical writing

Quality assurance

Tools

AI platforms

Job description

Medix seeks an AI Trainer & Evaluation Specialist – Healthcare for a 12-week contract, remote and flexible. You will design realistic clinical tasks and evaluate frontier AI models with your clinical judgment, focusing on accuracy, safety, and appropriate documentation.

You will build rubrics, flag potential failures, and compare model outputs to ensure professional standards are met, all while operating in a self-directed, long-form analytical role.