Remote AI Workflow Evaluator, Part-Time

OpenTrain AI; Northern (KY); Hybrid; USD 41,000 - 124,000; Part time

Job summary

OpenTrain AI is seeking an AI Agent Workflow Evaluator for remote, part-time work. You will test AI assistants (ChatGPT, Claude) in complex business workflows, scoring results with rubrics and recording detailed feedback.

The role focuses on clear written communication and accurate documentation of processes. Requires at least five years in a technology-enabled business function, a bachelor’s degree, and strong English writing.

Qualifications

• Minimum five years of professional experience in a business function where technology is used to solve operational challenges.
• Daily, hands-on professional use of ChatGPT, Claude, or both
• Experience creating, applying, or reviewing evaluation rubrics, QA scorecards, grading criteria, or content review guidelines
• Excellent written English and ability to document findings precisely and provide clear, actionable feedback

Responsibilities

• Reproduce authentic workplace use cases by connecting AI assistants with business tools
• Maintain accurate records so evaluations are transparent, consistent, and reproducible
• Score generated outputs using rubrics, QA scorecards, or grading criteria
• Write precise, actionable feedback about model performance
• Identify recurring strengths, weaknesses, and opportunities across workflows
• Connect AI assistants with tools like Google Drive, Gmail, Slack, and Notion
• Record each workflow step and compare AI behavior with operational requirements

Skills

Excellent written English

Document findings precisely

Using ChatGPT/Claude

Create/evaluate rubrics

QA scorecards

Education

Bachelor's degree or higher

Tools

ChatGPT

Claude

Google Drive

Gmail

Slack

Notion

Job description

OpenTrain AI is seeking an AI Agent Workflow Evaluator for remote, part-time work. You will test AI assistants (ChatGPT, Claude) in complex business workflows, scoring results with rubrics and recording detailed feedback.

The role focuses on clear written communication and accurate documentation of processes. Requires at least five years in a technology-enabled business function, a bachelor’s degree, and strong English writing.