Remote AI Evaluation Scenario Designer

Mindrift; United States; Remote; USD 40,000 - 80,000; Part time

Benefits offered by this job

Flexible work schedule

Competitive pay up to $80/hour

Valuable experience in AI projects

Job summary

A technology-driven company is seeking an entry-level professional to design evaluation scenarios for AI agents. Responsibilities include creating structured test cases and analyzing agent behaviors. Ideal candidates should have a degree in a relevant field, strong communication skills, and a basic understanding of programming. This part-time role offers flexible hours, with competitive pay rates up to $80/hour based on skills and experience. Contribute to advanced AI projects and enhance your portfolio while working from anywhere.

Qualifications

• Bachelor's and/or Master's Degree in Computer Science, Software Engineering, or related fields.
• Background in QA, software testing, data analysis, or NLP annotation.
• Can define expected agent behaviors and scoring logic.

Responsibilities

• Create structured test cases that simulate complex human workflows.
• Define gold-standard behavior and scoring logic to evaluate agent actions.
• Analyze agent logs and decision paths.

Skills

Good understanding of test design principles

Strong written communication skills in English

Basic experience with Python

Curious and open to working with AI-generated content

Education

Bachelor's and/or Master's Degree in relevant fields

Tools

JSON/YAML

Job description

A technology-driven company is seeking an entry-level professional to design evaluation scenarios for AI agents. Responsibilities include creating structured test cases and analyzing agent behaviors. Ideal candidates should have a degree in a relevant field, strong communication skills, and a basic understanding of programming. This part-time role offers flexible hours, with competitive pay rates up to $80/hour based on skills and experience. Contribute to advanced AI projects and enhance your portfolio while working from anywhere.