Remote AI Content Evaluator: Quality & Compliance

Dorado; Northern (KY); Hybrid; USD 45,000 - 65,000; Full time

Job summary

Innodata is seeking detail-oriented evaluators to conduct human quality evaluations for an enterprise AI customer support product on Instagram, WhatsApp, and Messenger. You will evaluate AI model responses against complex rubrics using provided business knowledge bases.

You’ll thrive with a customer service background, strong written English and French, and the ability to follow complex evaluation guidelines. This role uses web-based tools and labeling interfaces in a remote setting.

Qualifications

• Experience in customer service, call centers, retail, or handling customer communications via email/chat/phone.
• Exceptional written English with strong tone, brand voice, grammar and nuance.
• Exceptional written French with strong tone, brand voice, grammar and nuance.
• Strictly follow multi-tier evaluation guidelines, complex logic trees, and technical rubrics.
• Comfort using dedicated web-based tools and labeling interfaces.

Responsibilities

• Model Evaluation: Review and score AI-generated customer interactions across dimensions such as action fidelity, faithfulness, hallucination, compliance, tone, and handoff.
• Intent & Fact Verification: Benchmark queries against authoritative business sources within the task UI.
• Quality Assurance: Participate in dual-review processes and calibration audits to ensure inter-rater agreement.
• Performance Targets: Deliver precise evaluation targets and measurable outcomes.

Skills

Customer service background

English proficiency

French proficiency

Analytical precision

Tech adaptability

Job description

Innodata is seeking detail-oriented evaluators to conduct human quality evaluations for an enterprise AI customer support product on Instagram, WhatsApp, and Messenger. You will evaluate AI model responses against complex rubrics using provided business knowledge bases.

You’ll thrive with a customer service background, strong written English and French, and the ability to follow complex evaluation guidelines. This role uses web-based tools and labeling interfaces in a remote setting.