AI Agent Evaluation Engineer - Part-Time, Project-Based

Mindrift; Birmingham; On-site; GBP 42,030 - 59,862; Part time

Job summary

Mindrift is seeking experienced developers or software engineers for part-time, project-based roles evaluating AI coding agents. Responsibilities include creating tasks and evaluation criteria within simulated environments and reviewing AI-generated code. Ideal candidates will have a degree in Computer Science, over 5 years of development experience primarily in Python, and skills in full-stack development and test automation. Contributors can earn up to $50 per hour based on contribution pace.

Qualifications

• 5+ years in software development, primarily with Python.
• Background in full-stack development.
• Experience writing functional and integration tests.

Responsibilities

• Create tasks for evaluating AI coding agents.
• Assemble and calibrate tasks from virtual companies.
• Design tests that accept correct solutions and reject incorrect ones.

Skills

Python

React (JavaScript/TypeScript)

Docker

CI/CD (GitHub Actions)

Test automation

Education

Degree in Computer Science or related fields

Tools

Postgres

Kafka

Redis

Job description

Mindrift is seeking experienced developers or software engineers for part-time, project-based roles evaluating AI coding agents. Responsibilities include creating tasks and evaluation criteria within simulated environments and reviewing AI-generated code. Ideal candidates will have a degree in Computer Science, over 5 years of development experience primarily in Python, and skills in full-stack development and test automation. Contributors can earn up to $50 per hour based on contribution pace.