Senior AI Coding Agent Evaluation Engineer

Mindrift; South Africa; On-site; ZAR 753,285 - 1,072,861; Full time

Job summary

Mindrift in South Africa is seeking specialists for project-based opportunities in AI evaluation. You will create realistic developer environments to test AI coding agents, ensuring tasks challenge models effectively.

Successful candidates should have over 5 years of software development experience, particularly with Python and JavaScript. Proficiency in English and a knack for writing tests are essential. Compensation is project-based, up to $40/hr equivalent based on your expertise.

Qualifications

• 5+ years in software development.
• Experience with core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis.
• Proficiency in writing functional and integration tests.
• B2+ English proficiency required.

Responsibilities

• Create realistic developer environments with a codebase and infrastructure.
• Design tasks and evaluation criteria for AI coding agents.
• Write tests to verify agent solutions and iterate based on feedback.

Skills

Software Development

Python (FastAPI)

JavaScript/TypeScript (React)

Docker

Postgres

Kafka

Redis

Test Writing (functional, integration)

English Proficiency (B2+)

Job description

Mindrift in South Africa is seeking specialists for project-based opportunities in AI evaluation. You will create realistic developer environments to test AI coding agents, ensuring tasks challenge models effectively.

Successful candidates should have over 5 years of software development experience, particularly with Python and JavaScript. Proficiency in English and a knack for writing tests are essential. Compensation is project-based, up to $40/hr equivalent based on your expertise.