Python Engineer for AI Testing & Task Design

Mindrift; Sweden; On-site; SEK 803,265 - 1,004,081; Part time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies. You will design tasks that test safe, honest behavior in AI coding agents, write robust tests, and iterate based on QA feedback to ensure fair evaluation across diverse solutions.

As part of a hands-on engineering role, you will build realistic test environments, review agent submissions, and refine scenarios to maintain high-quality benchmarks for safety and performance.

Qualifications

• 4–5+ years in software development.
• Core stack: Python, JavaScript/TypeScript.
• Strong test design skills for safe vs unsafe evaluation.
• Hands-on experience with coding agents (Claude Code, GitHub Copilot CLI, Codex).
• Familiarity with GitHub PRs and CI workflows.
• English proficiency at least B2+.

Responsibilities

• Design tasks that pair a benign goal with unsafe shortcuts and write tests to catch it.
• Review agent solutions, analyze failures, and refine tasks for fair evaluation.
• Create realistic developer environments with codebases, infra, tickets, and docs.

Skills

Software development

Python

JavaScript/TypeScript

Test design

Coding agents

GitHub PRs & CI

English B2+

Tools

Claude Code

GitHub Copilot CLI

Codex

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies. You will design tasks that test safe, honest behavior in AI coding agents, write robust tests, and iterate based on QA feedback to ensure fair evaluation across diverse solutions.

As part of a hands-on engineering role, you will build realistic test environments, review agent submissions, and refine scenarios to maintain high-quality benchmarks for safety and performance.