211 Specialist AI Response Evaluator

OpenTrain AI, Inc.; Northern (KY); Hybrid; USD 41,000 - 55,000; Part time

Job summary

OpenTrain AI, Inc. is hiring for a remote, part-time contractor role to evaluate AI responses to questions about government services, public benefits, and community resources.

You will review sample questions, develop rubrics for accurate guidance, and assess AI answers against those rubrics to ensure relevance and usefulness for real callers. The position emphasizes clear written communication, sound judgment, and knowledge of public benefits and community resources, with experience in crisis

Qualifications

• Fluent English required.
• At least 1 year of regular, high-volume public contact by phone, chat, or in person.
• Background: 211 Specialist, Information and Referral Specialist, or Benefits Navigator experience.

Responsibilities

• Review sample questions and flag requests that do not reflect realistic public inquiries.
• Create rubrics that define accurate and complete answers, including referrals and next steps.
• Evaluate AI-generated answers against the rubrics.
• Identify errors, missing information, outdated guidance, and unhelpful advice.
• Provide clear written feedback asynchronously.

Skills

Public benefits

Community resources

Written communication

Judgment

Job description

The Work

You will help evaluate AI responses to questions about government services, public benefits, and community resources. Your work will focus on whether questions reflect real public needs and whether answers are accurate, complete, current, and useful to someone seeking help.

• Review sample questions and flag requests that do not reflect realistic public inquiries.
• Create rubrics that define accurate and complete answers, including referrals and next steps.
• Evaluate AI-generated answers against those rubrics.
• Identify errors, missing information, outdated guidance, and advice that would not help a real caller.
• Provide clear written feedback asynchronously.

What It Pays And Takes

This is a remote, asynchronous contractor engagement. The work is part time and focused on public-service content, benefits guidance, referrals, and AI-generated responses.

• Pay: $30 to $40 per hour.
• Location: Open to candidates in the United States.
• Language: Fluent English required.
• Experience: At least one year of regular, high-volume public contact by phone, chat, or in person.
• Required background: 211 Specialist, Information and Referral Specialist, or Benefits Navigator experience.
• Required skills: Working knowledge of public benefits and community resources, clear written communication, and sound judgment about the accuracy and usefulness of guidance.
• Helpful experience: Crisis intervention, needs assessment, community resource databases, or creating guides, scripts, and training materials.
• Helpful perspective: Experience serving diverse callers, including rural and Spanish-speaking communities.

About AI Training Work

AI training is the human work behind systems that answer questions and provide information. Contributors review and improve model responses so they are more accurate, complete, and useful, and people with relevant professional knowledge are especially well suited to this work. OpenTrain helps people find these projects, build a profile, and apply in one place.