AI Model And Agent Evaluator

OpenTrain AI, Inc.; United Kingdom; Remote; GBP 14,000 - 37,000; Part time

Job summary

OpenTrain AI, Inc. is seeking a remote, part-time contractor to test AI models and agents against defined scenarios and guidelines. You will perform repeatable evaluations, document findings in structured reports, and reset usage limits as required by the project process.

You should have experience with AI products, models, or AI agent testing, and able to follow complex instructions precisely with strong attention to detail in a high-volume setting.

Qualifications

• Experience with AI products, AI models, or AI agent evaluation and testing.
• Must follow complex instructions precisely and consistently.
• Maintain strong attention to detail during repetitive tasks.
• Comfortable working within tight project timelines.

Responsibilities

• Evaluate model and agent accuracy, reasoning, responsiveness, and instruction-following.
• Run repeatable tasks using detailed scenarios and project standards.
• Document findings and observations in structured written reports.
• Record findings and submit evaluation datasets through AI training platforms.

Skills

AI products experience

AI models

AI agent evaluation

Testing precision

Job description

The Work

You will test AI models and agents against set scenarios, project guidelines, and evaluation standards. Your work will help improve evaluation datasets and model quality.

The role includes repeatable testing, structured reporting, and careful review during high-volume evaluation activity.

• Evaluate model and agent accuracy, reasoning, responsiveness, and instruction-following.
• Run repeatable tasks using detailed scenarios and project standards.
• Document findings, observations, and unusual results in structured written reports.
• Reset usage limits and repeat evaluations when required by the project process.
• Record findings and submit evaluation datasets through AI training platforms.

What It Pays And Takes

This is a remote, part-time contractor role. It is open to candidates in India, Japan, and the United Kingdom, with English fluency required.

• Pay: $14 to $36 per hour.
• Location: Open to candidates in India, Japan, and the United Kingdom.
• Work type: Remote, part-time contractor engagement.
• Experience: Experience with AI products, AI models, or AI agent evaluation and testing is preferred.
• Follow complex instructions precisely and consistently.
• Maintain strong attention to detail during repetitive or high-volume tasks.
• Work comfortably within strict project timeframes.
• Helpful background: AI tools, software quality assurance, digital product testing, or AI research.

About AI Training Work

AI training is the human work behind artificial intelligence. Contributors test models, rate their responses, and prepare examples that help AI systems become more accurate, useful, and consistent.