Healthcare AI Evaluation Task Designer (Remote)

Patronus AI, Inc.; United States; Remote; USD 69,000 - 207,000; Part time

Job summary

Patronus AI, Inc. is seeking qualified clinicians and healthcare professionals to design and author realistic healthcare tasks for AI evaluation. You will create records, write task requests, solve tasks end-to-end, and define scoring criteria.

The role is remote, independent, and contract-based, with rolling start after a trial task and a commitment of 10+ hours per week. No AI background is required; you should be able to spot details in charts that affect care plans.

Qualifications

• 3+ years hands-on clinical or related work.
• Current or past licensure/certification in your field (MD/DO, NP, PA, RN, PharmD, CPC, CCS, CDIP, CCDS, CCM, CRCR).
• Knowledge of guidelines and documentation standards (ICD-10-CM/PCS, CPT, InterQual/MCG).
• Careful reading with chart-citation for every conclusion.
• Clear, precise professional English writing.
• Ability to work independently and asynchronously against deadlines.

Responsibilities

• Build fictional patient/operational records from a scenario.
• Write clear task requests with deliverables and handling of uncertainty.
• Solve tasks end-to-end and document a reference solution with reasoning.
• Write explicit scoring criteria for graders, including safe/unsafe answers.
• Run tasks against current models and adjust difficulty.
• Check accuracy and consistency of tasks and avoid shortcuts.

Skills

Years of hands-on experience

Licensure/certification

Reading and citing sources

Clear writing in professional English

Independent asynchronous work

Detail-oriented clinical observation

Education

MD/DO or NP/PA/RN/Pharmacist licensure

Specialty board credentials (CPHQ, CRCR, etc.)

Tools

Epic/Oracle Health/Solventum 360 Encompass

InterQual/MCG criteria

payer portals and claims systems

Job description

Patronus AI, Inc. is seeking qualified clinicians and healthcare professionals to design and author realistic healthcare tasks for AI evaluation. You will create records, write task requests, solve tasks end-to-end, and define scoring criteria.

The role is remote, independent, and contract-based, with rolling start after a trial task and a commitment of 10+ hours per week. No AI background is required; you should be able to spot details in charts that affect care plans.