AI Safety Annotation Reviewer

HumanitApp; United States; Remote; USD 70,000 - 110,000; Full time

Benefits offered by this job

Flexible working hours

Remote work environment

Competitive compensation

Job summary

Turing, a US-based AI services company, seeks a Team Lead to oversee a team of business analysts and guide quality reviews on policy misapplication, reasoning gaps, and annotation consistency for high-stakes AI tasks. You will enforce a detailed safety policy framework and provide constructive feedback to improve training and evaluation accuracy.

The role prioritizes rigorous analytical skills, clear communication, and adherence to US cultural norms, with remote-friendly 40-hour weeks and

Qualifications

• Demonstrated experience identifying systematic errors in human-generated annotations, evaluations, or written assessments.
• Ability to quickly understand and consistently apply detailed policy guidelines.
• Native-level English fluency and strong written communication skills.
• Comfort providing clear, direct, and constructive feedback.
• Strong understanding of US cultural context and communication norms.
• Ability to work independently, manage priorities, and maintain quality under time pressure.
• Availability to commit to 40 hours per week; immediate availability is preferred.

Responsibilities

• Audit trainer-submitted evaluations for policy misapplication, incomplete reasoning, scoring drift, schema errors, and annotation inconsistencies.
• Apply a detailed AI safety policy framework consistently across high-stakes review tasks.
• Identify individual and systematic errors in human-generated annotations and written assessments.
• Validate annotation completeness, scoring accuracy, policy alignment, and adherence to required schemas.
• Provide direct, constructive feedback to trainers and recommend corrective actions when quality issues are identified.
• Document recurring error patterns and contribute to quality improvement, calibration, and reviewer training initiatives.
• Maintain accuracy and consistency while working under time-sensitive conditions and handling sensitive content.

Skills

Attention to detail

Analytical reasoning

English fluency

Feedback delivery

Independent work

Job description

Future projects, not immediate vacancies. Check each network's country and experience requirements.

Turing is an AI-powered tech services company with a mission to accelerate AGI advancement and deployment by bridging the gap between global talent from 100+ countries and the world's best foundational LLM companies, by helping them improve performance through model evaluation, fine-tuning, feedback, factuality and handling diverse data.

Role Overview

A US based AI company that is at the forefront of AI research is looking for a Team Lead to oversee a team of business analysts. This opportunity is tailored for professionals who have experience in leading teams and has interest in supporting the development of solutions at the cutting‑edge of technology.

Job Responsibilities

Audit trainer‑submitted evaluations for policy misapplication, incomplete reasoning, scoring drift, schema errors, and annotation inconsistencies.

Apply a detailed AI safety policy framework consistently across high‑stakes review tasks.

Identify individual and systematic errors in human‑generated annotations and written assessments.

Validate annotation completeness, scoring accuracy, policy alignment, and adherence to required schemas.

Provide direct, constructive feedback to trainers and recommend corrective actions when quality issues are identified.

Document recurring error patterns and contribute to quality improvement, calibration, and reviewer training initiatives.

Maintain accuracy and consistency while working under time‑sensitive conditions and handling sensitive content.

Job Requirements

Sharp attention to detail and strong analytical and logical reasoning skills.

Demonstrated experience identifying systematic errors in human‑generated annotations, evaluations, or written assessments.

Ability to quickly understand and consistently apply detailed policy guidelines.

Native‑level English fluency and strong written communication skills.

Comfort providing clear, direct, and constructive feedback.

Strong understanding of US cultural context and communication norms.

Ability to work independently, manage priorities, and maintain quality under time pressure.

Availability to commit to 40 hours per week; immediate availability is preferred.

Preferred Qualifications

Experience in AI annotation review, Trust & Safety quality assurance, test engineering, content moderation, or editorial fact‑checking.

Previous experience with schema validation, policy compliance reviews, annotation completeness checks, or quality auditing.

Experience working with sensitive or high‑stakes content.

Importante Note: This project includes sensitive content involving topics such as self‑harm, violence, and abuse. Appropriate waivers, support resources, and wellbeing measures will apply.

Opportunity to work on cutting‑edge AI projects.

Competitive compensation.

Flexible working hours and remote work environment.

Compensation context

Compensation is not stated in the current feed. Confirm the rate and workload before accepting an assessment or project.

• Ability to quickly understand and consistently apply detailed policy guidelines.
• Ability to work independently, manage priorities, and maintain quality under time pressure.