AI Evaluators and Quality Experts, United States

Priceless Center LLC; United States; Hybrid; USD 45,000 - 67,000; Full time

Job summary

Priceless Center LLC in the United States is recruiting for a three-month remote AI evaluation project starting around October 13, 2026. We seek analytical, detail-oriented English-speaking evaluators for two tracks: Factuality Evaluation and Core Quality Evaluation.

You will review AI-generated responses, verify claims via credible sources, compare model outputs, and provide clear, evidence-based explanations following detailed guidelines.

Qualifications

• Based in the United States
• Fluent in English with strong reading and writing skills
• Ability to follow detailed instructions and multi-step workflows
• Experience in AI evaluation, data annotation, or QA is a plus

Responsibilities

• Review AI-generated responses across topics
• Identify verifiable claims and separate them for verification
• Research claims using credible online sources
• Determine whether claims are accurate, inaccurate, or unverifiable
• Compare outputs from multiple AI models to assess quality
• Evaluate accuracy, helpfulness, clarity, instruction-following and overall quality
• Provide clear, evidence-based explanations for evaluation decisions
• Follow detailed guidelines and maintain evaluation records

Skills

Analytical thinking

Attention to detail

Strong writing skills

Reading comprehension

Following instructions

Critical thinking

English proficiency

Research skills

Asking questions when unclear

Job description

AI Evaluators and Quality Experts, United States

AI, Data Annotation & Quality Assurance · United States · Full-time

Priceless Center LLC is recruiting qualified professionals based in the United States for a three-month remote AI evaluation project, with the earliest anticipated start date of October 13, 2026. We are seeking analytical, detail-oriented English-speaking experts for two specialized project tracks: Factuality Evaluation and Core Quality Evaluation. Factuality Evaluators will review AI-generated responses, verify individual claims through credible online sources, and assess factual accuracy. Core Quality Evaluators will compare responses from different AI models and assess accuracy, helpfulness, clarity, instruction-following, and overall quality. Candidates must meet the relevant project requirements, complete the required certification assessment, and achieve the required passing score before entering production.

Responsibilities

• Review AI-generated responses across a wide range of topics.
• Identify and separate individual claims that can be verified.
• Research claims using credible, authoritative online sources.
• Determine whether claims are accurate, inaccurate, or unverifiable.
• Compare responses from two or more AI models side by side.
• Evaluate accuracy, helpfulness, clarity, instruction-following, and overall response quality.
• Provide clear, evidence-based explanations for evaluation decisions.
• Follow detailed guidelines and multi-step workflows.
• Maintain consistent judgments and high-quality evaluation records.

Must be based in the United States. Fluent in English, with strong reading comprehension and writing skills. Strong critical-thinking and analytical abilities. Excellent attention to detail. Ability to understand and follow detailed instructions. Ability to complete the required certification assessment. For Factuality, strong web research and source-verification skills. For Core Quality, hands-on experience using large language models (LLMs) and evaluating their outputs.

Preferred qualifications

• Previous experience in AI evaluation, data annotation, or AI response rating.
• Experience with fact-checking, online research, or content quality assurance.
• Ability to compare AI responses across multiple quality dimensions.
• Experience working with multiple AI models and evaluation tools.
• Strong written reasoning and evidence-based decision-making skills.