AI Agent Tester Real-World Tasks - Remote

YO AI Labs

United States

Remote

USD 34,000 - 55,000

Full time

10 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

YO AI Labs is seeking U.S.-based AI Agent Testers to evaluate how AI assistants handle everyday tasks. Test two AI agents using your own accounts and smartphone, assess performance, and provide feedback to improve real-world usability.

Responsibilities include testing various tasks, approving agents at key points, screen-recording, and scoring outcomes. Strong attention to detail and excellent written communication are essential.

Qualifications

  • Must be based in the United States.
  • Bachelor's degree required.
  • 2–3 years of professional experience, preferably in engineering, business management, or operations.
  • Regular use of AI tools in daily life.
  • Strong attention to detail, written communication, and documentation skills.
  • Active personal AI assistant account (Plus), Muse and Dots access, screen-recording smartphone, and a payment card for reimbursable test purchases.
  • No coding or AI technical background required; generalists welcome.

Responsibilities

  • Test AI agents on tasks such as shopping, bookings, bill payments, returns, email follow-ups, travel, and online forms.
  • Approve or stop agents at important decision points.
  • Screen-record every task attempt.
  • Score performance based on completion, quality, user control, satisfaction, and time.
  • Provide clear written feedback on the overall user experience.

Skills

Attention to detail
Written communication
Documentation
Regular use of AI tools

Education

Bachelor's degree

Tools

Screen recording smartphone
Personal AI assistant account (Plus)
Muse access
Dots access

Job description

Job Title: AI Agent Tester – Real-World Tasks

Role Type: Contractor
Location: United States

Job Summary

We are seeking detail-oriented, U.S.-based AI Agent Testers to evaluate how AI assistants handle everyday tasks. You will test two AI agents using your own accounts and smartphone, assess their performance, and provide feedback to improve their real-world usability.

Key Responsibilities
  1. Test AI agents on tasks such as shopping, bookings, bill payments, returns, email follow-ups, travel, and online forms.
  2. Approve or stop agents at important decision points.
  3. Screen-record every task attempt.
  4. Score performance based on completion, quality, user control, satisfaction, and time.
  5. Provide clear written feedback on the overall user experience.
Required Qualifications
  1. Must be based in the United States.
  2. Bachelor's degree required.
  3. 2–3 years of professional experience, preferably in engineering, business management, or operations.
  4. Regular use of AI tools in daily life.
  5. Strong attention to detail, written communication, and documentation skills.
  6. An active personal AI assistant account (Plus tier), active Muse and Dots access, a smartphone with screen-recording capability, and a payment card for reimbursable test purchases.

No coding or AI technical background is required. Generalists are welcome, and domain experts are preferred.

Project Details
  • Kickoff: Project is already active.
  • Onboarding: Within 24–48 hours of selection.
  • Work Schedule: Flexible, entirely from your phone.
  • Task Duration: Approximately 1.5 hours per run, including evidence and scoring.
Selection Process
  1. Apply and complete the screening questions.
  2. Attend a 15–30-minute interview.
  3. Complete the profile review.
  4. Receive onboarding instructions from the Talent Operations team.
Why Join?
  • Work with AI tools and evaluate real-world performance.
  • Enjoy flexible, phone-based work.
  • Help shape and improve next-generation AI agents.

Originally posted on Himalayas

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Agent Tester - Real-World Tasks, Remote- Contract
AI Agent Tester - Real-World Tasks, Remote- Contract

Xperteez Technology • United States

Remote
USD 30,000 - 52,000
AI Agent Tester - Real-World Tasks
AI Agent Tester - Real-World Tasks

confidential • United States

Remote
USD 34,000 - 62,000
Everyday AI Agent Tester - Remote
Everyday AI Agent Tester - Remote

YO AI Labs • New York (NY)

Remote
USD 20,000 - 40,000
AI Agent Tester - Real-World Tasks
AI Agent Tester - Real-World Tasks

Remotebridge • United States

Remote
USD 34,000 - 62,000
Phone-based work
Flexible schedule
AI Agent Evaluator - Remote
AI Agent Evaluator - Remote

YO AI Labs • New York (NY)

Remote
USD 55,000 - 83,000
AI Agent Tester
AI Agent Tester

Frontier AI Lab • United States

Remote
USD 34,000 - 62,000
AI Agent Tester - Real-World Tasks
AI Agent Tester - Real-World Tasks

Micro1 • United States

Remote
USD 34,000 - 62,000
AI Agent Tester - Real-World Tasks
AI Agent Tester - Real-World Tasks

HumanitApp • United States

Remote
USD 34,000 - 62,000
AI Agent Evaluator
AI Agent Evaluator

micro1 Inc. • United States

Remote
USD 34,000 - 62,000
AI Agent Tester - Real-World Tasks
AI Agent Tester - Real-World Tasks

Remolity, Inc. • United States

Remote
USD 28,000 - 41,000