AI Evaluation Engineer: Build Real-World Coding Agent Tests

Mindrift; Doha; On-site; QAR 150,000 - 251,000; Part time

Benefits offered by this job

Mindrift project-based engagement

Job summary

Mindrift builds datasets to evaluate AI coding agents in realistic, simulated environments. You will craft tasks, evaluation criteria, and tests to ensure robust assessments of model performance across a believable development history.

We seek developers with 5+ years in software, strong Python/JS stacks, Docker and data tooling, plus English proficiency (B2+). This is a project-based role with flexible ownership and schedule.

Qualifications

• 5+ years in software development.
• Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis.
• Experience writing tests (functional, integration) and English B2+.

Responsibilities

• Build a dataset to evaluate AI coding agents in realistic environments.
• Create challenging tasks and evaluation criteria within simulated environments.
• Write tests that verify agent solutions, accepting all valid approaches.
• Iterate on tasks and tests based on QA feedback to ensure fairness and robustness.

Skills

5+ yrs dev

Python (FastAPI)

JavaScript/TypeScript

Testing: functional/integration

English B2+

Docker

Postgres

Kafka

Redis

Tools

Docker

Postgres

Kafka

Redis

Job description

Mindrift builds datasets to evaluate AI coding agents in realistic, simulated environments. You will craft tasks, evaluation criteria, and tests to ensure robust assessments of model performance across a believable development history.

We seek developers with 5+ years in software, strong Python/JS stacks, Docker and data tooling, plus English proficiency (B2+). This is a project-based role with flexible ownership and schedule.