Remote Agent Evaluation Engineer — AI Testing & Tasks

Mindrift; France; Remote; EUR 49,000 - 61,000; Part time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.

You're tasked with creating challenging tasks and evaluation criteria within realistic simulated environments: build developer environments, design tasks from intermediate states, and write tests that accept multiple valid solutions.

Qualifications

• 5+ years in software development
• Experience in QA-automation/testing or cybersecurity roles
• English proficiency B2+

Responsibilities

• Build realistic developer environments with codebase, infra, tickets and docs
• Design tasks from intermediate states of environments with clear success criteria
• Write tests that verify agent solutions and accept multiple valid approaches
• Iterate on tasks and tests based on QA feedback

Skills

Software development experience

Python (FastAPI)

JavaScript/TypeScript (React)

Docker

Postgres

Kafka

Redis

Testing (functional & integration)

English proficiency (B2+)

Education

Master’s Degree in Computer Science, Software Engineering, Data Science / Data Analytics, AI/ML, NLP, Information Systems or related fields

Bachelor’s degree with 5+ years experience

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.

You're tasked with creating challenging tasks and evaluation criteria within realistic simulated environments: build developer environments, design tasks from intermediate states, and write tests that accept multiple valid solutions.