Quality Analyst – US (AI Evaluation Project)

Codefeast; United States; On-site; USD 34,000 - 49,000; Part time

Job summary

Codefeast is seeking experienced Quality Analysts to support an AI evaluation project focused on reviewing and comparing responses from multiple chatbots. The role requires assessing outputs against guidelines and delivering clear final assessments for each task.

You will work independently, manage multiple evaluation tasks over a 16-week period, and ensure accuracy, relevance, and data quality. Immediate start; USA-only, short-term contract, $30/hour.

Qualifications

• 3+ years of experience in Quality Assurance, Data Annotation, Content Evaluation, Testing, or a similar role.
• Strong analytical, critical-thinking, and problem-solving skills.
• Comfortable working with AI tools, chatbots, and online platforms.

Responsibilities

• Evaluate and compare responses generated by multiple chatbots.
• Assess outputs for accuracy, relevance, completeness, clarity, and guideline adherence.
• Identify errors, inconsistencies, missing information, and subtle quality issues.
• Provide structured feedback and final assessments for each task.

Skills

Quality Assurance

Data Annotation

Content Evaluation

Tools

AI Tools

Chatbots

Job description

Experience: 3+ Years

Job Type: Short-Term Contract

Location: USA Only

Compensation: $30/hour

Start Date: Immediate

Domain: Quality Assurance | AI Evaluation | Chatbot Assessment | Data Quality

Role Overview

We are looking for experienced Quality Analysts / QA Analysts to support an AI evaluation project focused on reviewing and comparing responses generated by multiple chatbots.

The role involves assessing AI-generated outputs against defined guidelines, identifying errors or quality issues, and providing clear, structured final assessments for each evaluation task.

Key Responsibilities

• Evaluate and compare responses generated by multiple chatbots.
• Assess outputs for accuracy, relevance, completeness, clarity, and guideline adherence.
• Identify errors, inconsistencies, missing information, and subtle quality issues.
• Provide structured feedback and final assessments for each task.

Key Requirements

• 3+ years of experience in Quality Assurance, Data Annotation, Content Evaluation, Testing, or a similar role.
• Strong analytical, critical-thinking, and problem-solving skills.
• Comfortable working with AI tools, chatbots, and online platforms.
• Ability to work independently and manage multiple tasks.
• Must be based in the USA.

Preferred Skills

Prior experience with Chatbot Evaluation, LLM Testing, AI Response Evaluation, Data Quality, Model Evaluation, Content Review, or AI Training projects is an advantage.

Project Scope

• Project-based engagement involving a defined number of evaluation tasks.
• Each task involves comparing multiple chatbot responses and providing a final assessment.
• Candidates must be available full-time for the complete 16-week engagement.