Remote Freelance AI Evaluation Engineer (Paid per Task)

Mindrift; New South Wales; Remote; AUD 39,000 - 98,000; Part time

Benefits offered by this job

Remote work

Part-time freelance project

Job summary

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. This position is part-time, remote, and freelance.

You will design tasks, write tests that accept all valid solutions, and iterate based on QA feedback. The role emphasizes 5+ years of software development, core stack including Python (FastAPI), JavaScript/TypeScript, Docker, Postgres, Kafka, Redis.

Qualifications

• 5+ years in software development
• Core stack includes Python (FastAPI), JavaScript/TypeScript, Docker, Postgres, Kafka, Redis
• Experience writing tests (functional, integration)
• English proficiency - B2+ overall

Responsibilities

• Design tasks within realistic simulated environments
• Write tests that accept multiple valid solutions
• Evaluate agent solutions and QA feedback to refine tasks
• Iterate on tasks and tests based on reviewer feedback

Skills

Python (FastAPI)

JavaScript/TypeScript

Docker

Postgres

Kafka

Redis

Test automation

Education

Master’s degree in CS/SE/AI or related field

Bachelor’s degree with 5+ years experience

Job description

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. This position is part-time, remote, and freelance.

You will design tasks, write tests that accept all valid solutions, and iterate based on QA feedback. The role emphasizes 5+ years of software development, core stack including Python (FastAPI), JavaScript/TypeScript, Docker, Postgres, Kafka, Redis.