Remote AI Agent Evaluator: Test Real-Life Tasks

RemoteJobsOne; Houston (TX); Remote; USD 34,000 - 62,000; Full time

Job summary

RemoteJobsOne is seeking an AI Agent Evaluator for fully remote work in the United States. You will run everyday errands with two AI agents, record results on your phone, and provide detailed feedback.

The role requires a Bachelor's degree and 2–3 years of relevant experience in engineering, business management, or operations. A strong eye for detail and clear written communication are essential.

Qualifications

• US-based, with a Bachelor’s degree.
• 2–3 years of experience, preferably in engineering, business management or office/operations.
• Detail-oriented, great at following instructions and documenting each step.
• Clear written communication.
• Generalists welcome, domain experts preferred.
• No coding or AI background needed.

Responsibilities

• Run real-life scenarios: bills, bookings, shopping, travel and forms.
• Approve or stop the AI agent at key steps.
• Screen record every attempt.
• Score each run on completion, quality, control issues, satisfaction and time.
• Share clear written feedback on the agent experience.

Skills

Detail oriented

Clear written communication

Generalist mindset

Education

Bachelor's degree

Job description

RemoteJobsOne is seeking an AI Agent Evaluator for fully remote work in the United States. You will run everyday errands with two AI agents, record results on your phone, and provide detailed feedback.

The role requires a Bachelor's degree and 2–3 years of relevant experience in engineering, business management, or operations. A strong eye for detail and clear written communication are essential.