Business Analyst, AI Agent Evaluation

OpenTrain AI, Inc.; United States; Remote; USD 28,000 - 55,000; Part time

Benefits offered by this job

Part-time contractor

Job summary

OpenTrain AI, Inc. is seeking an entry-level contractor to evaluate how AI agents understand instructions, analyze information, reason through problems, and complete tasks in digital environments.

You will review AI-generated responses, perform online verification, and create evaluation scenarios for quality assessment. The role requires strong English communication, analytical thinking, and the ability to work 20+ hours weekly as a remote contractor.

Qualifications

• Review AI-generated responses for accuracy and task completion.
• Verify claims through online research and evidence evaluation.
• Solve analytical, data interpretation, arithmetic, and puzzle-based questions.
• Create questions and scenarios for evaluating LLM performance.
• Write clear annotations and explanations about response quality and errors.

Responsibilities

• Review AI responses and actions for accuracy and task completion.
• Verify claims through online research and assess supporting evidence.
• Solve logical reasoning, data interpretation, arithmetic, and puzzle-based questions.
• Create questions and scenarios for evaluating large language model performance.
• Write clear annotations and explanations about response quality and errors.

Skills

Strong English communication

Analytical reasoning

Research

Data interpretation

Critical thinking

Creative reasoning

Education

Study in engineering, literature, journalism, communications, arts, statistics, business or related discipline welcome

Tools

Excel

Google Sheets

Google Docs

Google Workspace

Job description

The Work

You will evaluate how computer-using AI agents understand instructions, analyze information, reason through problems, and complete tasks in digital environments. Your work will help assess the quality of AI-generated actions and responses.

You will analyze written content, research claims and sources online, solve analytical questions, and create evaluation scenarios. You will also rate responses, explain your decisions, and identify errors, missing information, inaccurate claims, and unsupported conclusions.

• Review AI-generated responses and actions for accuracy and task completion.
• Verify claims through online research and assess the supporting evidence.
• Solve logical reasoning, data interpretation, arithmetic, and puzzle-based questions.
• Create questions and scenarios for evaluating large language model performance.
• Write clear annotations and explanations about response quality and errors.

What It Pays And Takes

The listing does not provide a pay rate. This is an ongoing hourly freelance engagement with part-time contractor status.

The work is open worldwide and is intended for people who can work independently while following detailed instructions, quality standards, and timelines.

• Hours: 20 or more hours per week.
• Location: Open worldwide.
• Language: Strong English communication and reading comprehension.
• Experience level: Entry level.
• Engagement: Part-time freelance contractor work.
• Reasoning: Strong analytical, logical, critical, and creative reasoning skills.
• Research: Ability to evaluate evidence, identify reliable information, compare sources, and spot unsupported claims.
• Data skills: Ability to interpret data, identify patterns, and perform basic calculations.
• Helpful background: Business analysis, research, copywriting, journalism, technical writing, editing, translation, or a related field.
• Helpful tools: Excel, Google Sheets, Google Docs, and Google Workspace.
• Education: Study in engineering, literature, journalism, communications, arts, statistics, business, or a related discipline is welcome; equivalent skills and experience may also be considered.

About AI Training Work

AI training is the human work behind artificial intelligence, including reviewing model responses, checking information, and preparing examples that help systems perform better. OpenTrain helps people find these projects, build a profile, and grow a portfolio of AI training experience.