AI Medical Expert Evaluator

Dr. Hira Ali Shahid; Karachi Division; On-site; PKR 1,200,000 - 1,800,000; Full time

Job summary

Doctor seeks an AI Medical Expert Evaluator in Karachi to assess the accuracy, safety, and clinical relevance of AI-generated medical content. Apply medical expertise to structured evaluation tasks and help improve quality of healthcare AI systems.

Detail-oriented professional with sound clinical judgment is required. The evaluator collaborates with evaluation and product teams to clarify criteria and improve assessment workflows, while maintaining confidentiality and compliance with privacy and

Qualifications

• Strong medical knowledge to interpret clinical questions, terminology, and recommendations.
• Skill in checking medical claims for accuracy, relevance, completeness, and appropriate uncertainty.
• Sound judgment in identifying potentially harmful advice, contraindications, and safety concerns.
• Clear written communication, including concise explanations and structured annotation of findings.

Responsibilities

• Review AI-generated answers to medical questions for factual accuracy, clinical appropriateness, and clarity.
• Identify unsafe, incomplete, misleading, or unsupported recommendations and document specific concerns.
• Compare responses against trusted medical knowledge and evaluation guidelines.
• Rate and annotate outputs consistently, providing concise rationale and actionable feedback.
• Maintain confidentiality and follow applicable privacy, ethics, and data-handling requirements.
• Collaborate with evaluation and product teams to clarify clinical criteria and improve assessment workflows.

Skills

Medical knowledge

Clinical evaluation

Written communication

Attention to detail

Confidentiality & ethics

Tools

Web-based review tools

Digital documentation

Job description

Job Description

We are seeking an AI Medical Expert Evaluator in Karachi to assess the accuracy, safety, and clinical relevance of AI-generated medical content. In this role, you will apply medical expertise to structured evaluation tasks and help improve the quality of healthcare-related AI systems. The position is suited to a detail-oriented professional who can assess clinical information objectively and communicate findings clearly.

Key responsibilities

• Review AI-generated answers to medical questions for factual accuracy, clinical appropriateness, and clarity.
• Identify unsafe, incomplete, misleading, or unsupported recommendations and document specific concerns.
• Compare responses against trusted medical knowledge and evaluation guidelines.
• Rate and annotate outputs consistently, providing concise rationale and actionable feedback.
• Maintain confidentiality and follow applicable privacy, ethics, and data-handling requirements.
• Collaborate with evaluation and product teams to clarify clinical criteria and improve assessment workflows.

This is a Karachi-based, non-remote role. The employer is identified as Doctor; no further organizational details were provided. The evaluator contributes expert review rather than direct patient care, and should use sound clinical judgment while recognizing when a response requires additional context or specialist review.

Job Specification

• Strong medical knowledge and ability to interpret clinical questions, terminology, and recommendations.
• Skill in checking medical claims for accuracy, relevance, completeness, and appropriate uncertainty.
• Sound judgment in identifying potentially harmful advice, contraindications, and safety concerns.
• Clear written communication, including concise explanations and structured annotation of findings.
• High attention to detail and consistency when applying rubrics across repeated evaluation tasks.
• Ability to assess AI-generated text objectively without treating it as a substitute for professional care.
• Comfort with web-based review tools, digital documentation, and handling sensitive information responsibly.
• Professional integrity, discretion, and commitment to evidence-informed healthcare practice.