AI QA & Project Coordinator (Safety & Evaluation)

mpathic; Seattle (WA); On-site; USD 90,000 - 140,000; Full time

Job summary

mpathic seeks an experienced QA Reviewer & Project Coordinator to join our team in Seattle or Boston. This hands-on role combines expert evaluation and quality assurance with day-to-day project coordination, testing AI systems, and ensuring high-quality outputs while keeping projects organized and on schedule.

You will review AI content for safety and accuracy, conduct QA reviews, calibrate reviewer performance, and help improve workflows and documentation in a fast-paced startup environment.

Qualifications

• Experience reviewing AI safety work and quality assurance.
• Ability to document findings clearly and precisely.
• Strong organizational skills with attention to detail in fast-paced environments.
• Comfortable balancing multiple priorities and coordinating across teams.
• Experience with or exposure to AI systems, prompts, and red-teaming is a plus.

Responsibilities

• Review AI content for accuracy, safety, empathy, and policy compliance.
• Conduct QA reviews of expert evaluations and annotations.
• Calibrate reviewer performance to ensure consistency and quality.
• Improve QA workflows, reviewer documentation, and processes.
• Role-play clinical scenarios to evaluate AI behavior across situations.
• Perform red-teaming to identify risk and edge cases.
• Develop evaluation rubrics and scoring guidelines.
• Document inconsistencies and safety concerns for engineers.
• Collaborate with researchers to improve AI behavior with feedback.
• Maintain confidentiality with sensitive clinical content.
• Track project timelines, deliverables, and reviewer assignments.
• Coordinate review queues and balance workloads across teams.
• Identify risks, blockers, or quality issues early.
• Maintain project trackers and reporting dashboards.

Skills

Attention to detail

Excellent writing

Organizational skills

Multitasking

Independent worker

Cross-functional collaboration

Tools

Google Workspace

Slack

Spreadsheets

Project management tools

Job description

mpathic seeks an experienced QA Reviewer & Project Coordinator to join our team in Seattle or Boston. This hands-on role combines expert evaluation and quality assurance with day-to-day project coordination, testing AI systems, and ensuring high-quality outputs while keeping projects organized and on schedule.

You will review AI content for safety and accuracy, conduct QA reviews, calibrate reviewer performance, and help improve workflows and documentation in a fast-paced startup environment.