AI Coding Task Architect

Mindrift

South Carolina

On-site

USD 34,000 - 48,000

Full time

5 days ago
Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

Mindrift seeks a seasoned backend-focused engineer to craft realistic developer tasks for AI agents. You will design a complete task package including broken code, tests, instructions, and a reference solution, all inside a self-contained Docker environment.

You will write deterministic pytest tests, create a Jira-like instruction, and a solvable reference script, calibrating difficulty so state-of-the-art agents solve 20–60% of tasks. This is project-based, not permanent employment.

Qualifications

  • 3+ years of production software development in one backend stack.
  • Python plus pytest fluency required.
  • Experience authoring reproducible Docker environments with pinned dependencies.
  • Linux debugging inside containers (strace, lsof, journalctl).
  • AI coding agent experience and ability to cite past mistakes and fixes.
  • English writing at least B2+ level.

Responsibilities

  • Invent realistic developer scenarios with real bugs or missing features.
  • Build reproducible Docker environments with pinned dependencies.
  • Write deterministic pytest tests that verify outcomes without leaking fixes.
  • Create an instruction.md that reads like a Jira ticket for developers.
  • Develop a reference solve.sh proving the task is solvable.
  • Calibrate task difficulty so agents solve 20–60% of tasks.
  • Iterate based on expert QA feedback and later review other tasks.

Skills

Python
Docker
Linux / Bash
AI coding agent experience
English (B2+)

Tools

pytest
Dockerfile authoring

Job description

Mindrift seeks a seasoned backend-focused engineer to craft realistic developer tasks for AI agents. You will design a complete task package including broken code, tests, instructions, and a reference solution, all inside a self-contained Docker environment.

You will write deterministic pytest tests, create a Jira-like instruction, and a solvable reference script, calibrating difficulty so state-of-the-art agents solve 20–60% of tasks. This is project-based, not permanent employment.

Get your free, confidential resume review.
or drag and drop your file here.
Similar jobs

Similar jobs worth comparing

AI Coding Challenge Designer for Real-World Tasks
AI Coding Challenge Designer for Real-World Tasks

Socket.dev • Lebanon

On-site
USD 29,000 - 48,000
AI Task Design & Evaluation Engineer
AI Task Design & Evaluation Engineer

Mindrift • Michigan

On-site
USD 27,552,000 - 48,216,000
AI Coding Challenge Architect
AI Coding Challenge Architect

Mindrift • Egypt (PA)

On-site
USD 2,755,000 - 4,822,000
AI Coding Task Designer & Evaluation Specialist
AI Coding Task Designer & Evaluation Specialist

Mindrift • New York (NY)

On-site
USD 34,440,000 - 48,216,000
AI Evaluation Architect - Project-Based Code Agent Testing
AI Evaluation Architect - Project-Based Code Agent Testing

Mindrift • New York (NY)

On-site
USD 41,000 - 69,000
Flexible schedule
Project-based work
Senior AI Evaluation Engineer – Agent Testing & Tasks
Senior AI Evaluation Engineer – Agent Testing & Tasks

Dorado • United States

Remote
Senior Python Engineer — AI Task Designer & Evaluator
Senior Python Engineer — AI Task Designer & Evaluator

Dorado • United States

Remote
AI Coding Evaluator & Testing Architect (Python)
AI Coding Evaluator & Testing Architect (Python)

Mindrift • United States

On-site
USD 55,000 - 69,000
Project-Based Senior Software Engineer, AI Task Evaluation
Project-Based Senior Software Engineer, AI Task Evaluation

AI Chopping Block • Town of Poland (NY)

On-site
Senior Software Engineer - AI Task Designer (Project-Based)
Senior Software Engineer - AI Task Designer (Project-Based)

Dorado • United States

Remote