AI Agent Evaluator - Remote Smartphone Task Tester

OpenTrain AI, Inc.; Northern (KY); Remote; USD 34,000 - 62,000; Part time

Job summary

OpenTrain AI, Inc. is seeking a remote, part-time contractor to test two AI agents on a smartphone, completing realistic errands and recording outcomes. You will evaluate task completion, quality, control issues, user satisfaction, and time, then provide clear feedback.

The role requires a Bachelor's degree, strong written English, 2–3 years of relevant experience, and habitual use of AI tools. Flexible 1.5-hour runs with detailed documentation are expected.

Qualifications

• Bachelor's degree required.
• Two to three years of relevant experience.
• Regular use of AI tools in daily life.
• Strong attention to detail and reliable instruction-following.
• Clear written communication and organized documentation.
• Ability to record a smartphone screen.
• Active personal AI accounts that can be used on a phone.
• Generalists welcome; experience in related areas is helpful.
• No coding background or AI industry experience required.

Responsibilities

• Test two AI agents side by side on a smartphone.
• Record and score runs for completion, quality, control, satisfaction, and time.
• Write clear notes on what worked and what failed.
• Follow detailed instructions and document every step accurately.
• Approve or stop an agent when critical actions are needed.

Skills

Strong written English communication

Attention to detail

Instruction-following

Documentation skills

Education

Bachelor's degree

Tools

Screen recording on smartphone

Job description

OpenTrain AI, Inc. is seeking a remote, part-time contractor to test two AI agents on a smartphone, completing realistic errands and recording outcomes. You will evaluate task completion, quality, control issues, user satisfaction, and time, then provide clear feedback.

The role requires a Bachelor's degree, strong written English, 2–3 years of relevant experience, and habitual use of AI tools. Flexible 1.5-hour runs with detailed documentation are expected.