Senior Python Engineer - AI Task Designer & Evaluator

Mindrift; Sweden; On-site; SEK 798,609 - 2,395,827; Part time

Job summary

Mindrift is seeking experienced software developers to build and evaluate AI coding tasks for leading tech companies. You will create realistic environments, craft tasks from intermediate states, and write comprehensive tests to validate AI agent solutions.

This project-based role emphasizes quality and fairness, with flexible timing and an emphasis on English proficiency (B2+). You will work remotely on a global scale, designing innovative evaluation components for AI coding systems.

Qualifications

• 8+ years in software development.
• Experience writing tests (functional, integration).
• English proficiency – B2+.

Responsibilities

• Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history.
• Design tasks from intermediate states of these environments - craft the prompt, define what 'solved' means, and ensure the task is solvable by an AI agent.
• Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient.
• Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust.

Skills

8+ years in software development

Testing experience

English proficiency

Tools

Python (FastAPI)

JavaScript/TypeScript (React)

Docker

Postgres

Kafka

Redis

Job description

Mindrift is seeking experienced software developers to build and evaluate AI coding tasks for leading tech companies. You will create realistic environments, craft tasks from intermediate states, and write comprehensive tests to validate AI agent solutions.

This project-based role emphasizes quality and fairness, with flexible timing and an emphasis on English proficiency (B2+). You will work remotely on a global scale, designing innovative evaluation components for AI coding systems.