Remote Medical Assistant AI Evaluation Expert

Appsierra Group; United States; On-site; USD 77,000 - 96,000; Part time

Job summary

Appsierra Group is seeking an Experienced Medical Assistant Expert to create and refine healthcare reasoning challenges used to evaluate AI systems, applying hands-on medical knowledge to realistic clinical scenarios. You will develop questions on patient intake, vital signs, procedures, EHR documentation, and pharmacology, and assess AI responses for clinical accuracy and adherence to practice standards.

This remote contract role offers flexible, asynchronous work and the opportunity to

Qualifications

• Active CMA, RMA, CCMA, or equivalent Medical Assistant certification in good standing.
• Completion of an accredited Medical Assistant program or equivalent training.
• Proven clinical practice experience across clinical and administrative functions.
• Strong analytical, written, and verbal communication skills.
• Excellent attention to detail and ability to assess healthcare content accurately.

Responsibilities

• Develop clinical questions and case studies covering patient intake, vital signs assessment, medical procedures, EHR documentation, clinical laboratory procedures, pharmacology basics, and related medical assisting topics.
• Evaluate AI-generated responses for clinical accuracy and alignment with medical assisting standards of practice.
• Provide detailed, structured feedback identifying errors, gaps, and areas requiring improvement.
• Help calibrate question difficulty so evaluation content remains appropriately challenging and educational.
• Collaborate with healthcare and technical experts to refine clinical reasoning challenges and content quality.

Skills

Medical assisting

Analytical skills

Written and verbal communication

Attention to detail

Education

Accredited Medical Assistant program

Job description

Appsierra Group is seeking an Experienced Medical Assistant Expert to create and refine healthcare reasoning challenges used to evaluate AI systems, applying hands-on medical knowledge to realistic clinical scenarios. You will develop questions on patient intake, vital signs, procedures, EHR documentation, and pharmacology, and assess AI responses for clinical accuracy and adherence to practice standards.

This remote contract role offers flexible, asynchronous work and the opportunity to