AI Safety Annotation Reviewer

HumanitApp; Baton Rouge (LA); Remote; USD 80,000 - 110,000; Full time

Benefits offered by this job

Competitive pay

Flexible hours

Remote work environment

Job summary

Turing is seeking a Team Lead to oversee a team of business analysts focused on AI evaluation, annotation review, and policy compliance. You will guide quality assurance, calibrate scoring, and drive improvements across high-stakes tasks.

Ideal candidates have strong leadership, sharp analytical skills, and fluency in English, with the ability to provide clear feedback and manage priorities in a remote, fast-paced environment.

Qualifications

• Strong leadership and team management experience.
• Experience identifying systematic errors in annotations or evaluations.
• Ability to apply detailed policy guidelines consistently.
• Clear, constructive feedback skills.
• Excellent written communication and English fluency.

Responsibilities

• Audit trainer-submitted evaluations for policy misapplication, incomplete reasoning, scoring drift, schema errors, and annotation inconsistencies.
• Apply a detailed AI safety policy framework consistently across high-stakes review tasks.
• Identify individual and systematic errors in human-generated annotations and written assessments.
• Validate annotation completeness, scoring accuracy, policy alignment, and adherence to required schemas.
• Provide direct, constructive feedback to trainers and recommend corrective actions when quality issues are identified.
• Document recurring error patterns and contribute to quality improvement, calibration, and reviewer training initiatives.
• Maintain accuracy and consistency while working under time-sensitive conditions and handling sensitive content.

Skills

Team leadership

Quality assurance

Policy guidelines

Feedback delivery

Analytical reasoning

Tools

Policy review tools

Job description

About Turing

Turing is an AI-powered tech services company with a mission to accelerate AGI advancement and deployment by bridging the gap between global talent from 100+ countries and the world's best foundational LLM companies, by helping them improve performance through model evaluation, fine-tuning, feedback, factuality and handling diverse data.

Official role description

About Turing

Turing is an AI-powered tech services company with a mission to accelerate AGI advancement and deployment by bridging the gap between global talent from 100+ countries and the world's best foundational LLM companies, by helping them improve performance through model evaluation, fine-tuning, feedback, factuality and handling diverse data.

Role Overview

A US based AI company that is at the forefront of AI research is looking for a Team Lead to oversee a team of business analysts. This opportunity is tailored for professionals who have experience in leading teams and has interest in supporting the development of solutions at the cutting-edge of technology.

Job Responsibilities

• Audit trainer-submitted evaluations for policy misapplication, incomplete reasoning, scoring drift, schema errors, and annotation inconsistencies.
• Apply a detailed AI safety policy framework consistently across high-stakes review tasks.
• Identify individual and systematic errors in human-generated annotations and written assessments.
• Validate annotation completeness, scoring accuracy, policy alignment, and adherence to required schemas.
• Provide direct, constructive feedback to trainers and recommend corrective actions when quality issues are identified.
• Document recurring error patterns and contribute to quality improvement, calibration, and reviewer training initiatives.
• Maintain accuracy and consistency while working under time-sensitive conditions and handling sensitive content.

Job Requirements

• Sharp attention to detail and strong analytical and logical reasoning skills.
• Demonstrated experience identifying systematic errors in human-generated annotations, evaluations, or written assessments.
• Ability to quickly understand and consistently apply detailed policy guidelines.
• Native-level English fluency and strong written communication skills.
• Comfort providing clear, direct, and constructive feedback.
• Strong understanding of US cultural context and communication norms.
• Ability to work independently, manage priorities, and maintain quality under time pressure.
• Availability to commit to 40 hours per week; immediate availability is preferred.

Preferred Qualifications

• Experience in AI annotation review, Trust & Safety quality assurance, test engineering, content moderation, or editorial fact-checking.
• Previous experience with schema validation, policy compliance reviews, annotation completeness checks, or quality auditing.
• Experience working with sensitive or high-stakes content.
• Importante Note: This project includes sensitive content involving topics such as self-harm, violence, and abuse. Appropriate waivers, support resources, and wellbeing measures will apply.

Benefits

• Opportunity to work on cutting-edge AI projects.
• Competitive compensation.
• Flexible working hours and remote work environment.

Application Process

Shortlisted candidates will be reached out.

Relevant skills

Business AnalysisProject Management