Senior Software Engineer — AI Evaluation & Testing (Contract)

Mindrift; Poland; On-site; PLN 125,378 - 208,964; Full time

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. This is a project-based role, not permanent employment, with compensation tied to hourly work and deliverables.

You will help build a dataset to evaluate AI coding agents by creating realistic developer environments, crafting tasks, and writing robust tests that accept multiple correct approaches. English proficiency at B2+ is required.

Qualifications

• 5+ years in software development experience.
• Core stack includes Python (FastAPI), JavaScript/TypeScript (React).
• Experience writing tests (functional, integration).
• English proficiency at B2+ or higher.

Responsibilities

• Create challenging tasks and evaluation criteria for AI coding agent datasets.
• Build realistic developer environments with codebase, infra, tickets and docs.
• Design tasks from intermediate states and define solvable criteria.
• Write tests that accept all valid approaches and reject incorrect ones.
• Iterate tasks based on QA feedback to ensure fairness.

Skills

5+ years software dev

Python (FastAPI)

JavaScript/TypeScript (React)

Testing (functional, integration)

English B2+

Tools

Docker

Postgres

Kafka

Redis

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. This is a project-based role, not permanent employment, with compensation tied to hourly work and deliverables.

You will help build a dataset to evaluate AI coding agents by creating realistic developer environments, crafting tasks, and writing robust tests that accept multiple correct approaches. English proficiency at B2+ is required.