Remote AI Agent Evaluator (Contractor)

Zekai; United States; Remote; USD 34,000 - 62,000; Part time

Job summary

Zekai is seeking an AI Agent Evaluator to work remotely from the United States on a contractor basis. You will run real-life scenarios with two AI agents, screen record and score each run, and provide clear written feedback. A Bachelor's degree and the ability to operate AI tools in daily life are required.

You will evaluate agent performance across completion, quality, control issues, satisfaction, and time, helping shape the design of next-generation assistants.

Qualifications

• US-based with a Bachelor's degree.
• 2-3 years of experience in engineering, business management, or office/operations.
• Detail-oriented and strong at following instructions.

Responsibilities

• Run real-life scenarios: bills, bookings, shopping, travel, and errands.
• Approve or stop the agent at key steps.
• Screen record every attempt and score completion, quality, control, satisfaction, and time.

Skills

Detail-oriented

Written communication

Education

Bachelor's degree

Job description

Zekai is seeking an AI Agent Evaluator to work remotely from the United States on a contractor basis. You will run real-life scenarios with two AI agents, screen record and score each run, and provide clear written feedback. A Bachelor's degree and the ability to operate AI tools in daily life are required.

You will evaluate agent performance across completion, quality, control issues, satisfaction, and time, helping shape the design of next-generation assistants.