Remote Freelance AI Agent Evaluation Engineer

Visa Hunt; México; Presencial; MXN 702.000 - 936.000; Jornada completa

Descripción de la vacante

Mindrift is building a dataset to evaluate AI coding agents by creating realistic developer environments, tickets, docs, and conversations. The role involves crafting tasks from intermediate states of these environments and defining what counts as a solved solution.

This part-time, remote freelance opportunity pays per completed task, with rates up to $40 per hour depending on qualification tier and efficiency.

Formación

• 5+ years in software development experience.
• Proficiency with Python (FastAPI) and frontend JS/TS (React).
• Experience writing functional and integration tests.
• English proficiency at B2+.

Responsabilidades

• Create realistic developer environments with codebases and infrastructure.
• Design tasks from intermediate states and define success criteria.
• Write tests that accept multiple correct solutions and reject incorrect ones.
• Iterate on tasks and tests based on QA feedback.

Conocimientos

5+ years in software development

Python (FastAPI)

JavaScript/TypeScript (React)

Docker

PostgreSQL

Kafka

Redis

Testing (functional, integration)

English proficiency (B2+)

Educación

Master’s Degree in Computer Science or related

Bachelor’s degree + 5 years experience

Descripción del empleo

Mindrift is building a dataset to evaluate AI coding agents by creating realistic developer environments, tickets, docs, and conversations. The role involves crafting tasks from intermediate states of these environments and defining what counts as a solved solution.

This part-time, remote freelance opportunity pays per completed task, with rates up to $40 per hour depending on qualification tier and efficiency.