Remote AI Agent Evaluator: Test & Score Real-Life Tasks

RemoteJobsOne; Houston (TX); Remote; USD 34,000 - 62,000; Full time

Benefits offered by this job

Remote work

Job summary

RemoteJobsOne in the United States is hiring an AI Agent Evaluator on a fully remote, contractor basis. You’ll test two AI agents, run real-life scenarios, and rate completion, quality, and user experience from a real user perspective.

Requirements include a US-based status with a Bachelor’s degree, 2–3 years in related fields, strong attention to detail, and daily use of AI tools. You’ll work from your phone on your schedule and help shape next-generation AI agents.

Qualifications

• US-based with a Bachelor's degree and legal right to work in the United States.
• Applicants should already use AI tools in daily life.
• 2–3 years of experience in engineering, business management, or office/operations.
• Must be detail-oriented and able to document each step clearly.

Responsibilities

• Run real-life scenarios with two AI agents and record results.
• Approve or stop the agent at key steps.
• Screen record every attempt.
• Score runs on completion, quality, control issues, satisfaction, and time.
• Provide clear written feedback on the agent experience.

Skills

Detail-oriented

Instruction following

Daily AI usage

Education

Bachelor’s degree

Job description

RemoteJobsOne in the United States is hiring an AI Agent Evaluator on a fully remote, contractor basis. You’ll test two AI agents, run real-life scenarios, and rate completion, quality, and user experience from a real user perspective.

Requirements include a US-based status with a Bachelor’s degree, 2–3 years in related fields, strong attention to detail, and daily use of AI tools. You’ll work from your phone on your schedule and help shape next-generation AI agents.