Remote Freelance AI Evaluation Engineer

AI Chopping Block; Buenos Aires; Híbrido; ARS 31.166.000 - 62.331.000; A tiempo parcial

Ventajas ofrecidas por este puesto de trabajo

Flexible hours

Remote freelance project

Descripción de la vacante

Mindrift is seeking a seasoned software professional to design and evaluate AI developer tasks on a project-based basis. You will craft realistic environments, specify what counts as solved, and write robust tests to measure agent performance.

This part-time, remote freelance role emphasizes deep understanding of modeling failures and creating fair evaluation criteria across Python, React, and containerized tech stacks.

Formación

• 5+ years in software development.
• Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis.
• Experience writing tests (functional, integration).
• English proficiency - B2+.

Responsabilidades

• Create challenging tasks and evaluation criteria within realistic simulated environments.
• Write tests that verify agent solutions and accept multiple valid approaches.
• Iterate on tasks and tests based on QA feedback to ensure fairness.
• Review agent solutions, analyze failures, and refine prompts and tests.

Conocimientos

Software development

QA testing

English B2+

Task design

Educación

Master's Degree in Computer Science / Software Engineering

Bachelor’s degree (optional with 5 years experience)

Herramientas

Python (FastAPI)

JavaScript/TypeScript (React)

Docker

Postgres

Kafka

Redis

Descripción del empleo

Mindrift is seeking a seasoned software professional to design and evaluate AI developer tasks on a project-based basis. You will craft realistic environments, specify what counts as solved, and write robust tests to measure agent performance.

This part-time, remote freelance role emphasizes deep understanding of modeling failures and creating fair evaluation criteria across Python, React, and containerized tech stacks.