AI Safety Evaluator - Remote, High-Impact Model Quality

mercor; Portugal; Remote; EUR 74,000 - 86,000; Part time

Job summary

Mercor, a global talent network for AI research, is seeking an AI Safety Practitioner for a remote contract role. You will evaluate AI-generated responses for safety, accuracy, and policy compliance.

Responsibilities include reviewing content across sensitive domains, refining RLHF and SFT rubrics, and identifying unsafe outputs and reasoning failures.

You should have a Bachelor's degree or higher and 5+ years in AI safety or related fields, with excellent written English and analytical skills.

Qualifications

• Bachelor’s degree or higher in a related discipline.
• 5+ years in AI Safety, Trust & Safety, or related field.
• Excellent written English and strong analytical reasoning.
• Ability to assess nuanced, policy-sensitive scenarios.

Responsibilities

• Evaluate AI-generated responses for safety, factual accuracy, policy compliance, and overall quality.
• Review content involving misinformation, political persuasion, self-harm, violence, cyber, biosecurity, and other sensitive domains.
• Apply and refine evaluation rubrics for RLHF, SFT, and AI safety benchmarking.
• Identify unsafe outputs, hallucinations, reasoning failures, and policy violations.
• Provide structured feedback to improve model alignment and safety performance.
• Collaborate with AI researchers and safety teams on ongoing evaluation initiatives.

Skills

AI Safety experience

Excellent written English

Policy evaluation

Education

Bachelor's degree or higher

Job description

Mercor, a global talent network for AI research, is seeking an AI Safety Practitioner for a remote contract role. You will evaluate AI-generated responses for safety, accuracy, and policy compliance.

Responsibilities include reviewing content across sensitive domains, refining RLHF and SFT rubrics, and identifying unsafe outputs and reasoning failures.

You should have a Bachelor's degree or higher and 5+ years in AI safety or related fields, with excellent written English and analytical skills.