AI Task Architect for Backend Evaluation

Mindrift

Doha

On-site

QAR 150,000 - 201,000

Full time

14 days+
Application generator

Turn this role into an interview — a resume and cover letter built around what this employer wants.

Get past ATS filters

Job summary

Mindrift is seeking a seasoned backend-focused developer task designer. You will craft realistic coding challenges in Docker, assemble self-contained task packages, and write deterministic pytest-based tests to verify fixes. Deliverables include broken code, tests, instructions, and a reference solution.

This is project-based, not permanent employment. You will design tasks that resemble real-world developer work, ensuring reproducible environments, clear Jira-like tickets, and scalable

Qualifications

  • 3+ years of production software development in one backend stack.
  • Python + pytest fluency — required regardless of primary stack.
  • Experience with Docker, Linux and Bash in containerized environments.

Responsibilities

  • Invent realistic developer scenarios — real bugs, missing features, or broken ETL.
  • Build reproducible Docker environments with pinned dependencies.
  • Write a pytest-based verification that is deterministic and non-flaky.
  • Create an instruction.md that reads like a Jira ticket for a developer.
  • Provide a reference solve.sh proving the task is solvable.
  • Calibrate difficulty so state‑of‑the‑art agents solve the task 20–60% of the time.
  • Review other authors' tasks as a QA reviewer later.

Skills

Backend development
Python
pytest
Docker
Linux & Bash
AI tooling familiarity

Tools

Docker
pytest
CI/CD

Job description

Mindrift is seeking a seasoned backend-focused developer task designer. You will craft realistic coding challenges in Docker, assemble self-contained task packages, and write deterministic pytest-based tests to verify fixes. Deliverables include broken code, tests, instructions, and a reference solution.

This is project-based, not permanent employment. You will design tasks that resemble real-world developer work, ensuring reproducible environments, clear Jira-like tickets, and scalable

Get your free, confidential resume review.
or drag and drop your file here.
Similar jobs

Similar jobs worth comparing

Contract AI Evaluation Engineer - Design & Test Agent Tasks
Contract AI Evaluation Engineer - Design & Test Agent Tasks

Employment • Doha

On-site
QAR 150,000 - 251,000
AI Evaluation Engineer
AI Evaluation Engineer

Mindrift • Doha

On-site
QAR 150,000 - 251,000
Mindrift project-based engagement
AI Evaluation Engineer: Build Real-World Coding Agent Tests
AI Evaluation Engineer: Build Real-World Coding Agent Tests

Mindrift • Doha

On-site
QAR 150,000 - 251,000
Mindrift project-based engagement
Software Engineering Evaluation Specialist
Software Engineering Evaluation Specialist

Mindrift • Doha

On-site
QAR 150,000 - 201,000
Senior Python Engineer - AI Task Designer (Project-Based)
Senior Python Engineer - AI Task Designer (Project-Based)

Mindrift • Doha

On-site
QAR 301,000 - 1,003,000
Paid per task
AI Agent Evaluation Engineer
AI Agent Evaluation Engineer

Mindrift • Doha

On-site
QAR 207,000 - 295,000
Remote Full-Stack Web App Developer (AI Pilot)
Remote Full-Stack Web App Developer (AI Pilot)

Employment • Doha

On-site
QAR 248,000 - 354,000
Remote Full-Stack Web App Engineer (Part-Time)
Remote Full-Stack Web App Engineer (Part-Time)

Mindrift • Doha

On-site
QAR 201,000 - 301,000
Remote work
Flexible schedule
Performance bonus
AI Pilot: Mobile App Engineer (Remote, Part-Time)
AI Pilot: Mobile App Engineer (Remote, Part-Time)

Employment • Doha

On-site
QAR 201,000 - 301,000
Fully remote
Performance-based bonuses
Flexible hours
Remote Python Data Scraping Engineer (Part-Time)
Remote Python Data Scraping Engineer (Part-Time)

Mindrift • Doha

On-site
QAR 110,000 - 186,000