Senior AI Annotation Quality Lead (Remote)

OpenTrain AI, Inc.; United States; Remote; USD 117,000 - 255,000; Part time

Job summary

OpenTrain AI, Inc. is seeking a Senior Quality Lead for remote, part-time, project-based work. You will guide quality decisions across AI annotation and chatbot evaluation, turn guidelines into clear instructions, mentor reviewers, and drive calibration to improve training data quality.

The role emphasizes auditing against guidelines, documenting decisions, and optimizing reviewer performance. Five-day weeks with weekend coverage may be required; strong communication and leadership are essential.

Qualifications

• 4 to 6 years of overall professional experience, including at least 2 years leading QA reviewers.
• Strong judgment in complex or ambiguous cases and careful guideline-based auditing.
• Clear communication and strong documentation.

Responsibilities

• Validate project setup and test new task templates.
• Mentor QA Leads, annotation teams, and reviewers.
• Lead calibration sessions and resolve ambiguous cases.
• Audit work against detailed guidelines and document quality decisions.
• Analyze audit results and recommend improvements to guidelines and training.
• Identify high-performing reviewers for accelerated workflows.
• Compare multiple chatbots and provide final assessments for defined evaluation tasks.

Skills

Judgement in complex cases

Clear communication

Documentation

Job description

OpenTrain AI, Inc. is seeking a Senior Quality Lead for remote, part-time, project-based work. You will guide quality decisions across AI annotation and chatbot evaluation, turn guidelines into clear instructions, mentor reviewers, and drive calibration to improve training data quality.

The role emphasizes auditing against guidelines, documenting decisions, and optimizing reviewer performance. Five-day weeks with weekend coverage may be required; strong communication and leadership are essential.