AI Coding Agent Evaluator & Task Designer

Explore more jobs; Kuwait City; On-site; KWD 17,000 - 21,000; Full time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.

You'll build realistic developer environments, design tasks from intermediate states, write tests that verify agent solutions, and iterate on tasks based on QA feedback to ensure a fair and robust evaluation process.

Qualifications

• Minimum 5+ years in software development.
• Core stack includes Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis.
• Experience writing tests (functional, integration).
• English proficiency - B2+ required.

Responsibilities

• Create challenging tasks and evaluation criteria within realistic simulated environments.
• Design tasks from intermediate states of codebases and environments.
• Write tests that verify agent solutions and accommodate multiple valid approaches.
• Iterate on tasks and tests based on QA feedback to ensure fair and robust evaluation.

Skills

5+ years software development

Python

JavaScript/TypeScript

Docker

PostgreSQL

Kafka

Redis

Test writing (functional, integration)

English proficiency (B2+)

Tools

FastAPI

React

PostgreSQL

Kafka

Redis

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.

You'll build realistic developer environments, design tasks from intermediate states, write tests that verify agent solutions, and iterate on tasks based on QA feedback to ensure a fair and robust evaluation process.