Remote: Evaluation Scenario Designer for AI Agent Testing

Mindrift; À distance; EUR 40 000 - 60 000; Temps partiel

Résumé du poste

A leading AI innovation company is seeking candidates to design structured evaluation scenarios for LLM-based agents. The role involves creating test cases that simulate human-performed tasks, defining evaluation logic, and ensuring production readiness of scenarios. Requirements include a degree in a related field, strong analytical skills, and familiarity with coding formats. This flexible freelance project allows you to contribute from anywhere in the world, with pay rates up to $50/hour based on experience.

Prestations

Flexible scheduling

Competitive pay rates

Work from anywhere

Qualifications

• Bachelor’s and/or Master’s Degree in Computer Science, Software Engineering, or related fields.
• Good understanding of test design principles (e.g., reproducibility, coverage, edge cases).
• Strong written communication skills in English.

Responsabilités

• Create structured test cases that simulate complex human workflows.
• Define gold-standard behavior and scoring logic to evaluate agent actions.
• Analyze agent logs and failure modes.

Connaissances

Test design principles

Analytical mindset

Strong written communication in English

Basic experience with Python

Basic experience with JavaScript

Formation

Bachelor’s and/or Master’s Degree in Computer Science or related fields

Outils

JSON

YAML

Description du poste

A leading AI innovation company is seeking candidates to design structured evaluation scenarios for LLM-based agents. The role involves creating test cases that simulate human-performed tasks, defining evaluation logic, and ensuring production readiness of scenarios. Requirements include a degree in a related field, strong analytical skills, and familiarity with coding formats. This flexible freelance project allows you to contribute from anywhere in the world, with pay rates up to $50/hour based on experience.