Remote AI Evaluator — Shopping Assistant QA

Appsierra Group; American Samoa; On-site; USD 48,000 - 62,000; Full time

Benefits offered by this job

Fully remote

Weekly payment

Job summary

Appsierra Group is seeking AI Evaluators to assess the accuracy and usefulness of a digital shopping assistant. This independent contractor role is fully remote, with a workload of 20+ hours per week and compensation at $40 per hour.

You will review real user conversations, identify errors, and develop evaluation rubrics and verifiers to measure response quality across e-commerce interactions.

Qualifications

• Experience in data evaluation or quality assurance
• Experience in AI training or related field
• Familiarity with e-commerce search and digital shopping experiences
• Ability to analyze AI-generated responses critically
• Strong attention to detail and analytical skills

Responsibilities

• Review real user interaction traces with an AI shopping assistant.
• Identify inaccurate, illogical, incomplete, or unhelpful responses.
• Evaluate the quality and relevance of product recommendations.
• Analyze conversations to identify specific reasoning or response failures.
• Create structured rubrics for evaluating future responses.
• Develop verifiers to assess response quality against defined criteria.
• Complete evaluation work through the internal platform for 20+ hours per week.

Skills

Data evaluation

Quality assurance

AI training

E-commerce

Prompt engineering

Software testing

Job description

Appsierra Group is seeking AI Evaluators to assess the accuracy and usefulness of a digital shopping assistant. This independent contractor role is fully remote, with a workload of 20+ hours per week and compensation at $40 per hour.

You will review real user conversations, identify errors, and develop evaluation rubrics and verifiers to measure response quality across e-commerce interactions.