Remote AI Safety Lead & Annotation Quality Reviewer

HumanitApp; United States; Remote; USD 70,000 - 110,000; Full time

Benefits offered by this job

Flexible working hours

Remote work environment

Competitive compensation

Job summary

Turing, a US-based AI services company, seeks a Team Lead to oversee a team of business analysts and guide quality reviews on policy misapplication, reasoning gaps, and annotation consistency for high-stakes AI tasks. You will enforce a detailed safety policy framework and provide constructive feedback to improve training and evaluation accuracy.

The role prioritizes rigorous analytical skills, clear communication, and adherence to US cultural norms, with remote-friendly 40-hour weeks and

Qualifications

• Demonstrated experience identifying systematic errors in human-generated annotations, evaluations, or written assessments.
• Ability to quickly understand and consistently apply detailed policy guidelines.
• Native-level English fluency and strong written communication skills.
• Comfort providing clear, direct, and constructive feedback.
• Strong understanding of US cultural context and communication norms.
• Ability to work independently, manage priorities, and maintain quality under time pressure.
• Availability to commit to 40 hours per week; immediate availability is preferred.

Responsibilities

• Audit trainer-submitted evaluations for policy misapplication, incomplete reasoning, scoring drift, schema errors, and annotation inconsistencies.
• Apply a detailed AI safety policy framework consistently across high-stakes review tasks.
• Identify individual and systematic errors in human-generated annotations and written assessments.
• Validate annotation completeness, scoring accuracy, policy alignment, and adherence to required schemas.
• Provide direct, constructive feedback to trainers and recommend corrective actions when quality issues are identified.
• Document recurring error patterns and contribute to quality improvement, calibration, and reviewer training initiatives.
• Maintain accuracy and consistency while working under time-sensitive conditions and handling sensitive content.

Skills

Attention to detail

Analytical reasoning

English fluency

Feedback delivery

Independent work

Job description

Turing, a US-based AI services company, seeks a Team Lead to oversee a team of business analysts and guide quality reviews on policy misapplication, reasoning gaps, and annotation consistency for high-stakes AI tasks. You will enforce a detailed safety policy framework and provide constructive feedback to improve training and evaluation accuracy.

The role prioritizes rigorous analytical skills, clear communication, and adherence to US cultural norms, with remote-friendly 40-hour weeks and