Team Management AI Evaluator (English)

OpenTrain AI, Inc.; United States; Remote; USD 28,000 - 55,000; Part time

Job summary

OpenTrain AI, Inc. is hiring for a worldwide remote, part-time contract role focused on creating challenging prompts and evaluating language model responses.

Candidates should bring strong English writing, analytic thinking, and team-management experience to assess accuracy, reasoning, and usefulness of model outputs. The role requires leadership and performance management skills, workforce planning, and operational execution to judge AI handling of management topics, with a flexible, remote,

Qualifications

• Master's degree or higher required.
• At least three years of professional, research, or teaching experience.

Responsibilities

• Create complex prompts that test large language model capabilities.
• Evaluate AI-generated responses for factual accuracy, reasoning, relevance, and overall quality.
• Find factual errors, logical inconsistencies, knowledge gaps, and other weaknesses.
• Design challenging test cases and edge cases that reveal model limitations.
• Write clear, detailed, evidence-based feedback to support measurable model improvement.

Skills

Analytical thinking

Critical thinking

Attention to detail

Feedback writing

Education

Master's degree or higher

Tools

Prompt engineering

AI evaluation

Model testing

Job description

Create challenging prompts and evaluate large language model responses for accuracy, reasoning, and usefulness. This worldwide, part-time role requires strong English writing, analytical thinking, and team management experience.

100% Remote

Contract, part-time

Engagement

Remote

Location

Oct 23, 2026

Posted

The work

You will help test and improve large language models by creating difficult prompts, reviewing their responses, and explaining where they fall short. Your feedback will focus on whether responses are accurate, logical, relevant, and useful in practice.

The role uses your background in team leadership, people management, performance management, workforce planning, and operational execution to assess how well AI handles management-related topics.

• Create complex prompts that test large language model capabilities.
• Evaluate AI-generated responses for factual accuracy, reasoning, relevance, and overall quality.
• Find factual errors, logical inconsistencies, knowledge gaps, and other weaknesses.
• Design challenging test cases and edge cases that reveal model limitations.
• Write clear, detailed, evidence-based feedback to support measurable model improvement.

What it pays and takes

This is a part-time contractor role with a commitment of 20 or more hours per week. Pay details are not provided in the role information.

• Language: Professional fluency in English, including excellent written communication.
• Schedule: Part time, 20 or more hours per week.
• Experience level: Entry level.
• Education: A master's degree or higher in a relevant field.
• Experience: At least three years of professional, research, or teaching experience.
• Skills: Strong analytical and critical-thinking abilities, exceptional attention to detail, and the ability to challenge assumptions and find subtle factual or reasoning errors.
• Helpful background: Experience with large language models, generative AI, prompt engineering, AI evaluation, model testing, edge-case design, or stress-testing AI models.

About AI training work

AI training is the human work behind systems that generate and understand text. Evaluators review model responses, identify problems, and provide examples that help improve accuracy, reasoning, and reliability.

Requirements

• Experience: Entry level