Remote Part-Time Evaluation Scenario Writer for AI Testing

Mindrift; Town of Texas (WI); Remote; USD 83,000 - 110,000; Part time

Benefits offered by this job

Flexibility in work schedule

Competitive rates up to $80/hour

Opportunity to work on advanced AI projects

Job summary

A leading AI innovation firm is seeking an Evaluation Scenario Designer to create structured test cases for LLM-based agents. This part-time role allows for flexible project work, focusing on defining gold-standard behaviors and analyzing agent performance. Candidates should hold a degree in Computer Science or related fields and possess strong analytical skills. Experience in software testing and communication in English are essential. This opportunity offers competitive rates up to $80/hour based on skills and experience.

Qualifications

• Bachelor’s and/or Master’s Degree in relevant fields required.
• Good understanding of test design principles necessary.
• Strong written communication skills are a must.

Responsibilities

• Create structured test cases that simulate complex human workflows.
• Define gold‑standard behavior to evaluate agent actions.
• Analyze agent logs, failure modes, and decision paths.

Skills

Test design principles

Strong written communication skills in English

Basic experience with Python

Basic experience with JavaScript

Analytical mindset

Education

Bachelor’s or Master’s Degree in Computer Science

Related fields (Data Science, AI, NLP)

Job description

A leading AI innovation firm is seeking an Evaluation Scenario Designer to create structured test cases for LLM-based agents. This part-time role allows for flexible project work, focusing on defining gold-standard behaviors and analyzing agent performance. Candidates should hold a degree in Computer Science or related fields and possess strong analytical skills. Experience in software testing and communication in English are essential. This opportunity offers competitive rates up to $80/hour based on skills and experience.