Remote AI Evaluation Scenarios Writer & Test Designer

Mindrift; Australia; Remote; AUD 41,328 - 61,992; Part time

Benefits offered by this job

Flexible project work hours

Rates up to $45/hour

Experience with advanced AI projects

Job summary

A forward-thinking AI company is seeking an entry-level QA Analyst to design evaluation scenarios for AI-based agents. This part-time, flexible remote opportunity involves creating structured test cases, defining gold-standard behavior, and analyzing agent logs. Applicants should have a relevant degree and a keen analytical mindset. The rate can reach up to $45/hour, making it ideal for those looking to influence the future of AI while working around existing commitments.

Qualifications

• Bachelor's and/or Master's Degree in related fields.
• Experience in software testing or data analysis.
• Good understanding of test design principles.

Responsibilities

• Create structured test cases for complex workflows.
• Analyze agent logs and decision paths.
• Ensure scenarios are production-ready and reusable.

Skills

Analytical mindset

Attention to detail

Strong written communication in English

Basic experience with Python

Basic experience with JavaScript

Education

Bachelor's and/or Master's Degree in Computer Science or related fields

Background in QA or data analysis

Tools

JSON

YAML

Job description

A forward-thinking AI company is seeking an entry-level QA Analyst to design evaluation scenarios for AI-based agents. This part-time, flexible remote opportunity involves creating structured test cases, defining gold-standard behavior, and analyzing agent logs. Applicants should have a relevant degree and a keen analytical mindset. The rate can reach up to $45/hour, making it ideal for those looking to influence the future of AI while working around existing commitments.