AI Agent Evaluation Engineer — Test Architect

Mindrift; Philippines; On-site; PHP 412,061 - 824,123; Part time

Job summary

Mindrift is looking for specialists to create tasks and evaluation criteria for AI coding agents. The goal is to design realistic developer environments and ensure tasks are genuinely challenging for AI models.

The role requires over 5 years of software development experience, proficiency in Python and JavaScript/TypeScript, and strong English skills at the B2+ level. Compensation is project-based, up to $30/hr depending on experience.

Qualifications

• 5+ years in software development.
• Experience working with core stack: Python, JavaScript/TypeScript, Docker.
• Ability to write functional and integration tests.
• Strong English proficiency at B2+ level.

Responsibilities

• Build realistic developer environments for AI coding evaluation.
• Design tasks and criteria to challenge AI coding agents.
• Write tests to verify AI solutions.
• Iterate on tasks based on feedback from QA.

Skills

Software development

Python (FastAPI)

JavaScript/TypeScript (React)

Docker

Postgres

Kafka

Redis

Writing tests

English proficiency (B2+)

Job description

Mindrift is looking for specialists to create tasks and evaluation criteria for AI coding agents. The goal is to design realistic developer environments and ensure tasks are genuinely challenging for AI models.

The role requires over 5 years of software development experience, proficiency in Python and JavaScript/TypeScript, and strong English skills at the B2+ level. Compensation is project-based, up to $30/hr depending on experience.