Public Services AI Response Evaluator

OpenTrain AI, Inc.; Northern (KY); Hybrid; USD 41,000 - 55,000; Part time

Job summary

OpenTrain AI, Inc. is hiring for a remote, part-time contractor role to evaluate AI responses to questions about government services, public benefits, and community resources.

You will review sample questions, develop rubrics for accurate guidance, and assess AI answers against those rubrics to ensure relevance and usefulness for real callers. The position emphasizes clear written communication, sound judgment, and knowledge of public benefits and community resources, with experience in crisis

Qualifications

• Fluent English required.
• At least 1 year of regular, high-volume public contact by phone, chat, or in person.
• Background: 211 Specialist, Information and Referral Specialist, or Benefits Navigator experience.

Responsibilities

• Review sample questions and flag requests that do not reflect realistic public inquiries.
• Create rubrics that define accurate and complete answers, including referrals and next steps.
• Evaluate AI-generated answers against the rubrics.
• Identify errors, missing information, outdated guidance, and unhelpful advice.
• Provide clear written feedback asynchronously.

Skills

Public benefits

Community resources

Written communication

Judgment

Job description

OpenTrain AI, Inc. is hiring for a remote, part-time contractor role to evaluate AI responses to questions about government services, public benefits, and community resources.

You will review sample questions, develop rubrics for accurate guidance, and assess AI answers against those rubrics to ensure relevance and usefulness for real callers. The position emphasizes clear written communication, sound judgment, and knowledge of public benefits and community resources, with experience in crisis