Remote AI Agent Evaluation Specialist

Mindrift; Milano; Remoto; EUR 30.000 - 48.000; Part-time

Descrizione del lavoro

Mindrift connects specialists with project-based AI opportunities for leading tech companies. You will contribute to building realistic developer environments, crafting AI evaluation tasks, and writing robust tests.

This is a part-time, remote freelance role focused on QA automation and evaluation of AI coding agents. Ideal candidates have 5+ years in software development and a core stack including Python (FastAPI), React, Docker, Postgres, Kafka, and Redis, with strong testing experience and

Competenze

• 5+ years of professional software development experience.
• Core stack includes Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis.
• Experience writing tests (functional and integration) to validate solutions.
• English proficiency at B2+ level.

Mansioni

• Design tasks from intermediate states of simulated environments.
• Write tests that accept multiple correct solutions and reject incorrect ones.
• Evaluate agent solutions and provide QA feedback for fairness and robustness.
• Iterate on tasks and tests based on feedback to ensure solid evaluation.

Conoscenze

5+ years in software development

Python (FastAPI)

JavaScript/TypeScript (React)

Docker

Postgres

Kafka

Redis

Testing (functional, integration)

English proficiency (B2+)

Formazione

Master’s Degree in Computer Science / related field

Bachelor’s degree accepted with 5 years experience

Strumenti

Python (FastAPI)

React

Docker

Postgres

Kafka

Redis

Descrizione del lavoro

Mindrift connects specialists with project-based AI opportunities for leading tech companies. You will contribute to building realistic developer environments, crafting AI evaluation tasks, and writing robust tests.

This is a part-time, remote freelance role focused on QA automation and evaluation of AI coding agents. Ideal candidates have 5+ years in software development and a core stack including Python (FastAPI), React, Docker, Postgres, Kafka, and Redis, with strong testing experience and