Remote AI Agent Evaluation Scenarios Designer

Mindrift; Milano; Remoto; EUR 30.000 - 50.000; Part-time

Descrizione del lavoro

A technology firm is seeking a professional to design structured evaluation scenarios for LLM-based agents. The role involves creating test cases that simulate human tasks and defining expected behaviors for agents. Ideal candidates will have a background in computer science and experience in QA or data analysis. This is a fully remote freelance position, offering rates up to $32/hour based on skills and experience. Flexible scheduling allows for work that fits around other commitments.

Servizi

Flexible freelance schedule

Competitive hourly rates

Remote work opportunities

Competenze

• Bachelor's and/or Master’s in Computer Science, Software Engineering, Data Science, AI, NLP, or similar.
• Background in QA, data analysis, or NLP annotation is required.
• Good understanding of test design principles like reproducibility and edge cases.

Mansioni

• Design structured test scenarios for LLM-based agents.
• Define acceptable agent behaviors and scoring logic.
• Annotate task steps and expected outputs.

Conoscenze

Attention to detail

Analytical mindset

Good written communication in English

Familiarity with JSON/YAML

Curiosity about AI

Formazione

Bachelor's or Master’s Degree in related fields

Strumenti

Python

JavaScript

Descrizione del lavoro

A technology firm is seeking a professional to design structured evaluation scenarios for LLM-based agents. The role involves creating test cases that simulate human tasks and defining expected behaviors for agents. Ideal candidates will have a background in computer science and experience in QA or data analysis. This is a fully remote freelance position, offering rates up to $32/hour based on skills and experience. Flexible scheduling allows for work that fits around other commitments.