Trust & Safety Analyst – AI Evaluation - English

Welocalize; Delhi; On-site; INR 393,000 - 524,000; Full time

Job summary

Welocalize seeks detail-oriented Trust & Safety Analysts – AI Evaluation to review and evaluate AI-generated outputs using content safety policies, project guidelines, and expert human judgment.

You will assess complex AI outputs, provide accurate evaluations, and help improve the safety and quality of AI systems. Strong analytical skills and consistency with guidelines are essential.

Qualifications

• Educational background in Trust & Safety or related fields.
• Experience with content safety policies and AI evaluation.
• Ability to apply detailed policies to complex content.

Responsibilities

• Review AI outputs against content safety policies and guidelines.
• Apply expert judgment to context-dependent cases.
• Grade and label AI outputs consistently and accurately.
• Consider language, intent, and cultural differences in evaluations.
• Flag unclear cases and document decisions with rationale.
• Provide high-quality feedback to support AI training.

Skills

Trust & Safety

Content Safety

Policy

Linguistics

Research

Data annotation

Attention to detail

English writing

Education

Bachelor's degree or equivalent

Tools

AI evaluation tools

Job description

Job Overview

We are seeking detail-oriented Trust & Safety Analysts – AI Evaluation to review and evaluate AI-generated outputs using content safety policies, project guidelines, and expert human judgment.

In this role, you will assess complex AI outputs and provide accurate, consistent evaluations that help improve the safety and quality of AI systems. You will review situations where context, language, intent, and cultural understanding are important in determining the appropriate evaluation.

An ideal candidate has strong analytical skills, sound judgment, and experience applying content safety policies consistently. You should be comfortable reviewing complex or ambiguous content and making decisions based on defined guidelines rather than personal opinion.

Your work will provide high-quality human feedback and evaluation data that supports the training and improvement of AI systems.

Project Details

• Contract Type: Freelance, with the potential to convert to a full-time role.
• Pay Rate: US$3 per hour
• Location: India
• Language: English

Responsibilities

• Review and evaluate AI-generated outputs according to defined content safety policies and project guidelines.
• Apply expert human judgment to assess complex, ambiguous, or context-dependent situations.
• Grade, classify, label, or annotate AI outputs accurately and consistently.
• Consider context, language, intent, and cultural differences when evaluating content.
• Identify potential content safety or policy concerns based on defined guidelines.
• Evaluate difficult cases and make informed decisions when the appropriate outcome depends on multiple factors.
• Provide accurate human feedback to support AI system training and evaluation.
• Identify unclear or unusual cases and flag them according to defined project processes.
• Document decisions and supporting reasoning clearly when required.
• Apply detailed project guidelines consistently across assigned tasks.
• Maintain high levels of quality, accuracy, consistency, and attention to detail.

Required Qualifications

• Educational background or equivalent experience in Trust & Safety, Content Safety, Policy, Linguistics, Communications, Journalism, Research, Law, Social Sciences, or a related field.
• Experience in trust and safety, content moderation, content policy, AI evaluation, data annotation, quality assurance, or a related field.
• Strong understanding of content safety concepts and the ability to apply detailed policies and guidelines.
• Experience reviewing or evaluating complex content where context and intent are important.
• Ability to apply consistent judgment while separating personal opinions from defined policies and project requirements.
• Awareness of cultural and linguistic differences and their impact on content interpretation.
• Strong analytical and critical-thinking skills.
• Ability to make informed decisions in complex or ambiguous situations.
• Strong written English comprehension and communication skills.
• Excellent attention to detail and the ability to maintain consistency across high volumes of work.
• Familiarity with AI-generated content, AI evaluation, human feedback, RLHF, or data annotation is preferred.