Remote AI Evaluation Engineer for Code Tasks

YO AI Labs; California (MO); Remote; USD 50,000 - 80,000; Part time

Job summary

YO AI Labs invites a Senior Software Engineer to help build reinforcement learning environments that test AI models on complex coding tasks using MCP tools. You design reproducible environments, verify deterministically, and craft reference solutions for bug fixes, features, refactoring, and optimization.

No prior AI experience required. Responsibilities include creating environments, building verification systems, and evaluating agents on real-world codebases.

Qualifications

• Experience building reinforcement learning environments for software tasks.
• Ability to design reproducible environments and deterministic verification.
• Proven multi-language coding experience and strong problem solving.
• Strong written and verbal communication, able to work remotely.

Responsibilities

• Create reinforcement learning environments for software tasks.
• Design tasks involving bug fixing, feature development, refactoring, performance optimization.
• Build deterministic verification systems and golden reference solutions.
• Evaluate AI agents' reasoning on complex codebases and MCP tools.
• Develop realistic, reproducible software engineering scenarios.
• Document solutions and provide clear technical feedback.

Skills

Python 3

Java

Rust

C++

TypeScript

Algorithms

Data structures

Bug fixing

Debugging

Feature development

Code refactoring

Performance optimization

Communication

Attention to detail

Job description

YO AI Labs invites a Senior Software Engineer to help build reinforcement learning environments that test AI models on complex coding tasks using MCP tools. You design reproducible environments, verify deterministically, and craft reference solutions for bug fixes, features, refactoring, and optimization.

No prior AI experience required. Responsibilities include creating environments, building verification systems, and evaluating agents on real-world codebases.