Contract AI Evaluation Engineer - Design & Test Agent Tasks

Employment; Doha; On-site; QAR 150,000 - 251,000; Part time

Job summary

Mindrift is building a dataset to evaluate AI coding agents by creating challenging tasks and evaluation criteria within realistic simulated environments. You will design tasks from intermediate states, craft prompts, and define what constitutes a solvable solution.

As part of this project-based opportunity, you will write tests, review agent solutions, and iterate on tasks based on QA feedback. This role emphasizes collaboration with leading tech companies and requires strong English

Qualifications

• 5+ years in software development.
• Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis.
• Experience writing tests (functional, integration).
• English proficiency - B2+

Responsibilities

• Design tasks for AI coding agents within realistic simulated environments.
• Craft prompts and define what counts as a solvable solution.
• Write tests that verify agent solutions and accept multiple valid approaches.
• Iterate on tasks and tests based on QA feedback to ensure fairness and robustness.

Skills

Python

FastAPI

JavaScript/TypeScript

Docker

PostgreSQL

Kafka

Redis

Testing

Tools

Docker

PostgreSQL

Kafka

Redis

Job description

Mindrift is building a dataset to evaluate AI coding agents by creating challenging tasks and evaluation criteria within realistic simulated environments. You will design tasks from intermediate states, craft prompts, and define what constitutes a solvable solution.

As part of this project-based opportunity, you will write tests, review agent solutions, and iterate on tasks based on QA feedback. This role emphasizes collaboration with leading tech companies and requires strong English