AI Coding Agent Evaluator - Design & QA Real-World Tasks

Mindrift; Durban; On-site; ZAR 740,283 - 1,054,342; Part time

Job summary

Mindrift is looking for seasoned software developers to engage in project-based opportunities focused on evaluating AI coding agents. The successful candidates will create realistic development environments and design tasks that challenge AI agents.

Applicants should have at least 5 years of experience in software development, well-versed in Python, JavaScript/TypeScript, and Docker. This position offers flexibility in work hours and compensation of up to $40/hr, allowing developers to choose their schedule.

Qualifications

• 5+ years in software development.
• Experience with core stack including Python, JavaScript/TypeScript, and Docker.
• Proven experience writing functional and integration tests.
• Fluency in English at B2 level or higher.

Responsibilities

• Build realistic developer environments for AI evaluation.
• Design tasks and write tests to verify agent solutions.
• Iterate on tasks based on feedback to ensure robust evaluations.

Skills

Software development (5+ years)

Python (FastAPI)

JavaScript/TypeScript (React)

Docker

Postgres

Kafka

Redis

Writing tests (functional, integration)

English proficiency (B2+)

Job description

Mindrift is looking for seasoned software developers to engage in project-based opportunities focused on evaluating AI coding agents. The successful candidates will create realistic development environments and design tasks that challenge AI agents.

Applicants should have at least 5 years of experience in software development, well-versed in Python, JavaScript/TypeScript, and Docker. This position offers flexibility in work hours and compensation of up to $40/hr, allowing developers to choose their schedule.