Contract AI Evaluation Engineer (Python/Docker)

Dorado; United States; Remote; USD 34,440,000 - 48,216,000; Full time

Job summary

Mindrift is seeking a skilled software developer to design practical, containerized coding tasks for AI agents. You will create self-contained Docker environments, write deterministic tests, and supply instructions and a reference solution proving solvability.

Responsibilities include crafting realistic scenarios, building reproducible environments, and ensuring tests are robust and non-flaky. This is project-based work with competitive hourly compensation and strong emphasis on quality and QA

Qualifications

• 3+ years of production software development in one backend stack.
• Python + pytest fluency required irrespective of primary stack.
• Docker authoring with reproducible environments and pinned dependencies.
• Linux and Bash proficiency for debugging inside containers and shells.
• Experience with AI coding agents (e.g., Claude Code, Cursor) and ability to cite past failures.

Responsibilities

• Invent a realistic developer scenario with a real bug, broken ETL, or missing feature.
• Build a reproducible Docker environment with pinned dependencies.
• Write a deterministic pytest that verifies outcomes without leaking fixes.
• Create an instruction.md that reads like a Jira ticket for a developer.

Skills

Python

pytest fluency

AI coding agent experience

English (B2+)

Linux debugging

Tools

Docker

Bash

Job description

Mindrift is seeking a skilled software developer to design practical, containerized coding tasks for AI agents. You will create self-contained Docker environments, write deterministic tests, and supply instructions and a reference solution proving solvability.

Responsibilities include crafting realistic scenarios, building reproducible environments, and ensuring tests are robust and non-flaky. This is project-based work with competitive hourly compensation and strong emphasis on quality and QA