Remote Freelance: AI Agent Evaluation Specialist

Mindrift; United States; Remote; USD 17,000 - 28,000; Full time

Benefits offered by this job

Fully remote freelance

Competitive $20/hour rate

Flexible part-time schedule

Job summary

Mindrift is seeking a detail‑oriented professional to design realistic evaluation scenarios for LLM-based agents. You will craft test cases, define gold-standard behavior, and ensure each scenario is clear, scored, and reusable. Strong analytical mindset and academic background in CS/AI are required.

Join a fully remote freelance role that accommodates flexible hours. English at advanced level (C1+) is essential, and you should be ready to learn new methods while collaborating with developers to

Qualifications

• Bachelor's or Master's degree in Computer Science, Software Engineering, Data Science / AI, NLP, or related field.
• Advanced English language skills (C1 or higher).
• Willingness to learn new methods and handle complex guidelines.

Responsibilities

• Design structured test scenarios based on real-world tasks.
• Define golden path and acceptable agent behavior.
• Annotate task steps, expected outputs, and edge cases.
• Collaborate with developers to test scenarios and improve clarity.
• Review agent outputs and adapt tests accordingly.

Skills

Test design

LLM evaluation

Analytical thinking

Attention to detail

English (C1+)

Education

Bachelor's or master's in CS/AI

Job description

Mindrift is seeking a detail‑oriented professional to design realistic evaluation scenarios for LLM-based agents. You will craft test cases, define gold-standard behavior, and ensure each scenario is clear, scored, and reusable. Strong analytical mindset and academic background in CS/AI are required.

Join a fully remote freelance role that accommodates flexible hours. English at advanced level (C1+) is essential, and you should be ready to learn new methods while collaborating with developers to