Remote Senior AI Evaluation Engineer – Software Tasks

YO AI Labs

Miami (FL)

Remote

USD 96,000 - 152,000

Part time

2 days ago

Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

YO AI Labs is seeking experienced Senior Software Engineers to support an AI training project by building reinforcement learning environments that test AI models on complex software tasks using MCP tools.

You will design reproducible environments, ensure deterministic verification, and prepare reference solutions for bug fixing, feature work, refactoring, and performance optimization. This is a remote contractor position with ~15 hours/week.

Qualifications

Senior Software Engineer with strong Python, Java, Rust, C++, or TypeScript expertise.
Solid understanding of algorithms and data structures.
Proven ability to fix complex software issues and debug effectively.
Experience implementing features and refactoring large codebases.
Strong focus on performance tuning and optimization.
Excellent written and verbal communication skills.
High attention to detail and thorough problem solving.

Responsibilities

Create reinforcement learning environments for software engineering tasks.
Design tasks involving bug fixing, feature development, refactoring, and performance optimization.
Build deterministic verification systems and golden reference solutions.
Evaluate AI agents’ ability to reason through complex codebases using MCP tools.
Develop realistic, reproducible software engineering scenarios.
Ensure tasks accurately measure coding ability, problem-solving, and tool use.
Document solutions and provide clear technical feedback.