Business Analyst Computer Use Agent Evaluation

OpenTrain AI, Inc.; United States; Remote; USD 25,000 - 39,000; Part time

Job summary

OpenTrain AI, Inc. is seeking a part-time contractor to help evaluate AI agents. You will examine how well an AI system understands instructions, reasons through problems, researches information, and completes tasks in digital environments.

You will review AI-generated responses, verify claims with online sources, explain accuracy, and craft questions and scenarios to test large language models.

Qualifications

• Excellent English communication and reading comprehension.
• Entry-level opportunity; previous experience in listed field is helpful but not required.
• Helpful experience in research, copywriting, journalism, or technical writing.
• Experience with online research and evaluating claims is a plus.

Responsibilities

• Analyze, summarize, and organize complex written information.
• Research claims, sources, and supporting evidence online.
• Solve analytical, logical reasoning, data interpretation, and puzzle-based questions.
• Rate AI responses and actions, then provide annotations, explanations, and feedback.
• Create scenarios and questions for evaluating large language models.

Skills

Analytical thinking

Logical reasoning

Creative thinking

Data interpretation

Attention to detail

Education

Bachelor's degree or equivalent in a related field

Tools

Excel

Google Sheets

Google Docs

Google Workspace

Job description

The work

You will help evaluate computer-using AI agents. The work involves checking how well an AI system understands instructions, reasons through problems, researches information, and completes tasks in digital environments.

You will review AI-generated responses and actions, check claims and sources through online research, and explain whether the work is accurate and complete. You will also create questions and scenarios that help test and improve large language models.

• Analyze, summarize, and organize complex written information.


• Research claims, sources, and supporting evidence online.


• Solve analytical, logical reasoning, data interpretation, arithmetic, and puzzle-based questions.


• Rate AI responses and actions, then provide annotations, explanations, and feedback.


• Create scenarios and questions for evaluating large language models.

What it pays and takes

The project is an hourly, part-time contract with an initial 16-week duration and possible extension. The role is open worldwide, and the work requires at least 20 hours each week.

• Pay: The listing does not provide a rate.


• Hours: 20 or more hours per week.


• Location: Open worldwide.


• Language: Excellent English communication and reading comprehension.


• Experience: Entry-level opportunity; previous experience in a listed field is helpful but not required.


• Thinking skills: Analytical, logical, critical, and creative thinking, plus basic calculations and data interpretation.


• Work style: Ability to work independently while following detailed instructions, quality standards, and timelines.


• Preferred education: A degree in engineering, literature, journalism, communications, arts, statistics, business, or a related field.


• Helpful experience: Business analysis, research, copywriting, journalism, technical writing, editing, translation, or similar work.


• Helpful tools: Excel, Google Sheets, Google Docs, and Google Workspace.

About AI training work

AI training work is the human side of building artificial intelligence. People review examples, test model responses, and explain what is accurate, useful, or incorrect so AI systems can improve.