Remote Clinical AI Evaluator & Task Designer

Planet Pharma; San Francisco (CA); Remote; USD 120,000 - 180,000; Full time

Job summary

Planet Pharma is seeking experienced clinicians to evaluate frontier AI models in real clinical work. You will design realistic tasks such as SOAP notes, reconciliations, triage decisions, discharge summaries, patient education, or protocols, and run them through frontier AI agents to assess performance against professional standards.

You will work with real-world clinical files you assemble yourself, ranging from compact to multi-day scenarios, aiming to push AI to meet expert-level judgments

Qualifications

• Active or recent clinical license (RN, NP, PA, MD/DO, PharmD or allied health).
• 2+ years hands-on clinical practice preferred; qualified regardless of experience with practical work.
• Able to draw on real-world experience and day-to-day workflows to craft scenarios.
• Hands-on practitioner, not only managerial roles.
• Excellent written and spoken English at a professional level.
• Familiarity with AI/LLM tools and ability to judge AI outputs.
• Baseline tech literacy: Google Workspace and basic file handling.
• Available at least 10 hours per week; no weekly maximum.
• Based in United States, Canada, or the UK.

Responsibilities

• Design realistic clinical tasks drawn from day-to-day practice.
• Run tasks through frontier AI models and evaluate outputs against professional standards.
• Compare model outputs and document performance gaps.
• Write grading rubrics detailing deliverables and reasoning.
• Flag concrete failures with evidence and reasoning.
• Refine tasks and review tasks created by other clinicians.

Skills

Clinical practice

English proficiency

Attention to detail

Critical thinking

Documentation literacy

AI/LLM familiarity

Education

Bachelor’s degree or higher

Tools

Google Workspace

AI/LLM tools (e.g., Claude, ChatGPT)

Job description

Planet Pharma is seeking experienced clinicians to evaluate frontier AI models in real clinical work. You will design realistic tasks such as SOAP notes, reconciliations, triage decisions, discharge summaries, patient education, or protocols, and run them through frontier AI agents to assess performance against professional standards.

You will work with real-world clinical files you assemble yourself, ranging from compact to multi-day scenarios, aiming to push AI to meet expert-level judgments