Senior Quality Lead (AI Annotation)

OpenTrain AI, Inc.; United States; A distancia; USD 117.000 - 255.000; A tiempo parcial

Descripción de la vacante

OpenTrain AI, Inc. is seeking a Senior Quality Lead for remote, part-time, project-based work. You will guide quality decisions across AI annotation and chatbot evaluation, turn guidelines into clear instructions, mentor reviewers, and drive calibration to improve training data quality.

The role emphasizes auditing against guidelines, documenting decisions, and optimizing reviewer performance. Five-day weeks with weekend coverage may be required; strong communication and leadership are essential.

Formación

• 4 to 6 years of overall professional experience, including at least 2 years leading QA reviewers.
• Strong judgment in complex or ambiguous cases and careful guideline-based auditing.
• Clear communication and strong documentation.

Responsabilidades

• Validate project setup and test new task templates.
• Mentor QA Leads, annotation teams, and reviewers.
• Lead calibration sessions and resolve ambiguous cases.
• Audit work against detailed guidelines and document quality decisions.
• Analyze audit results and recommend improvements to guidelines and training.
• Identify high-performing reviewers for accelerated workflows.
• Compare multiple chatbots and provide final assessments for defined evaluation tasks.

Conocimientos

Judgement in complex cases

Clear communication

Documentation

Descripción del empleo

Lead quality decisions for AI annotation and chatbot evaluation work, mentor reviewers, resolve difficult cases, and improve guidelines in a remote 16-week project.

100% Remote

Contract, part-time

Engagement

Remote

Location

Oct 30, 2026

Posted

The Work

As a Senior Quality Lead, you will guide quality decisions across AI annotation and chatbot evaluation projects. You will turn detailed guidelines into clear instructions, improve reviewer consistency, and help strengthen how AI training data is produced and assessed.

The listing title identifies this role as a Senior Quality Lead, while the duties focus on AI annotation quality. Review the duties and qualifications before applying.

• Validate project setup and test new task templates.
• Mentor QA Leads, annotation teams, and reviewers.
• Lead calibration sessions and resolve ambiguous cases.
• Audit work against detailed guidelines and document quality decisions.
• Analyze audit results and recommend improvements to guidelines and training.
• Identify high-performing reviewers for accelerated workflows.
• Compare multiple chatbots and provide final assessments for defined evaluation tasks.

What It Pays and Takes

The role is a remote, project-based freelance engagement. It is structured as a contractor and part-time opportunity.

• Pay: Not specified in the provided role details.
• Schedule: 20+ hours per week.
• Working pattern: Five working days per week, including weekends as required, with two weekdays off.
• Working hours: 7:00 a.m. to 4:00 p.m. Pacific Time.
• Location: Open worldwide and fully remote.
• Experience: 4 to 6 years of overall professional experience, including at least 2 years leading QA reviewers.
• Skills: Strong judgment in complex or ambiguous cases, careful guideline-based auditing, clear communication, and strong documentation.
• Helpful background: Annotation, data labeling, AI training data, multimodal projects, quality management, QA, team management, calibration, and reviewer mentoring.

About AI Training Work

AI training is the human work behind modern artificial intelligence. Contributors label data, review model responses, evaluate chatbots, and provide feedback so AI systems can learn from reliable examples.

Requirements

• Experience: Entry level

Create and review difficult materials science problems for AI training, with clear solutions and grading criteria. This fully remote contractor role pays $85 to $185 per hour and offers flexible part-time work.

Create and review difficult engineering and physical science problems used to evaluate AI systems. This flexible remote contractor role pays $85 to $185 per hour and is open to experienced technical experts worldwide across listed countries.

Create and review difficult manufacturing and industrial engineering problems for AI evaluation, with pay of $85 to $185 per hour. This flexible remote contractor role is open across many countries.

Create and assess difficult semiconductor engineering problems for AI training, with $85 to $185 per hour for experienced technical judgment. Work remotely as a flexible contractor.