AI Evaluator - Healthcare - Fully Remote

Grayde.ai; India; Remote; INR 827,000 - 1,653,000; Full time

Job summary

Grayde.ai is seeking medical professionals to review and improve healthcare AI outputs. This remote role involves rating AI-generated drug information responses, verifying dosing and interactions, and ensuring proper citing from reputable sources.

Candidates will assess off-label handling, flag potential misinformation, and write structured rationales. Degree in medicine or pharmacy is required, and the work supports better clinical AI systems.

Qualifications

• Professional medical degree(s) required (MBBS/MBChB, MD, BPharm/MPharm, PharmD, MN/MSN).

Responsibilities

• Rank AI-generated drug information responses by quality using a structured rubric.
• Verify dosing, interactions, and contraindication accuracy.
• Assess evidence or labeling citations in responses.
• Use drug information databases (DrugBank, Micromedex, Lexicomp, FDA labeling) to validate information.
• Check handling of off-label and unapproved-use questions.
• Flag responses that could mislead prescribers or patients.
• Apply the rubric consistently across ranking batches.
• Write a comparative rationale explaining each preference judgment.

Education

MBBS / MBChB, MD, BPharm / MPharm, PharmD, MN / MSN (Nursing)

Job description

Grayde is a specialised healthcare and life science AI data and evaluation company. We work with the best clinicians and life science researchers to make medical AI models better.

Overview

Rank pairs of AI-generated responses as RLHF preference data, judging clinical accuracy, completeness, and appropriate sourcing against a scoring rubric.

About this job

We're seeking medical professionals - dentists, pharmacists, radiologists, nurses, oncologists etc to review and improve healthcare AI.

This is remote work which includes rating AI output, choosing the right answer, fact-checking, and writing training data.

What you'll do

1. Rank multiple AI-generated drug information responses by quality to produce RLHF preference pairs.
2. Verify dosing, interaction, and contraindication accuracy.
3. Assess whether responses appropriately cite evidence or labelling.
4. Use drug information databases such as DrugBank, Micromedex, Lexicomp, and FDA labelling to validate medication information.
5. Check handling of off-label and unapproved-use questions.
6. Flag responses that could mislead a prescriber or patient.
7. Apply a structured rubric consistently across ranking batches.
8. Write a comparative rationale explaining each preference judgment.

Who can apply

Degree

MBBS / MBChB (Primary medical), MD (Doctor of Medicine), BPharm / MPharm, PharmD, MN / MSN (Nursing)