Remote AI Agent Evaluation Analyst

Mindrift; Minnesota; Remote; USD 55,104 - 110,208; Part time

Benefits offered by this job

Rates up to $80/hour

Flexible remote work

Valuable experience on AI projects

Job summary

A leading AI consultancy is seeking QA professionals for autonomous AI agents focused on validating complex task structures. This flexible, project-based opportunity is ideal for analysts and students looking to utilize their analytical skills. The role involves reviewing evaluation tasks, identifying inconsistencies, and defining behaviors for AI agents. Ideal candidates will have strong attention to detail and experience with scenario design. Competitive rates up to $80/hour based on expertise.

Qualifications

• Ability to assess complex scenarios holistically.
• Experience with structured scenario design preferred.
• Exposure to AI-generated content is a plus.

Responsibilities

• Review evaluation tasks for logic and completeness.
• Identify inconsistencies or unclear decision points.
• Define expected behaviors for AI agents.

Skills

Excellent analytical thinking

Strong attention to detail

Good communication and clear writing in English

Familiarity with structured data formats

Experience with policy evaluation or logic puzzles

Education

Background in consulting, academia, or research

Job description

A leading AI consultancy is seeking QA professionals for autonomous AI agents focused on validating complex task structures. This flexible, project-based opportunity is ideal for analysts and students looking to utilize their analytical skills. The role involves reviewing evaluation tasks, identifying inconsistencies, and defining behaviors for AI agents. Ideal candidates will have strong attention to detail and experience with scenario design. Competitive rates up to $80/hour based on expertise.