Remote AI Agent Evaluation Engineer - Part-Time

Mindrift; Lisbon (AR); Remote; USD 27,552,000 - 55,104,000; Part time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies. This is a remote, freelance, part-time engagement focused on testing, evaluating, and improving AI systems within realistic simulated environments.

You will build developer environments, design tasks, write tests, and iterate based on QA feedback. Required: 5+ years dev experience, strong Python/React/Docker/Postgres/Kafka/Redis, and English at least B2+.

Qualifications

• Master’s Degree in Computer Science or related field is required.
• Bachelor’s degree accepted with 5 years of experience in the field.
• Minimum 3 years in related QA-automation/testing or cybersecurity roles.

Responsibilities

• Build realistic developer environments with codebase, infrastructure, and context.
• Design tasks and define what counts as solved for AI agents.
• Write tests that accept all valid solutions and reject incorrect ones.
• Iterate on tasks and tests based on QA feedback to improve fairness and robustness.

Skills

5+ years software development

English proficiency B2+

Python (FastAPI)

JavaScript/TypeScript (React)

Docker

Postgres

Kafka

Redis

Education

Master’s Degree in Computer Science / related field

Bachelor’s degree with 5 years experience

Tools

QA-automation tools

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies. This is a remote, freelance, part-time engagement focused on testing, evaluating, and improving AI systems within realistic simulated environments.

You will build developer environments, design tasks, write tests, and iterate based on QA feedback. Required: 5+ years dev experience, strong Python/React/Docker/Postgres/Kafka/Redis, and English at least B2+.