Remote AI Agent Evaluation Engineer

YO AI Labs; Los Angeles (CA); Remote; USD 83,000 - 138,000; Part time

Job summary

YO AI Labs seeks experienced Senior Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using MCP tools.

You will design reproducible environments, deterministic verification, and reference solutions for tasks such as bug fixing, feature implementation, codebase refactoring, and performance optimization. No prior AI experience is required.

Qualifications

• Strong proficiency in Python 3, Java, Rust, C++, or TypeScript.
• Strong understanding of algorithms and data structures.
• Experience with bug fixing and debugging complex software issues.
• Proven experience in feature implementation and codebase refactoring.
• Strong knowledge of performance optimization and tuning.
• Excellent written and verbal communication.
• Strong attention to detail.

Responsibilities

• Create reinforcement learning environments for software engineering tasks.
• Design tasks involving bug fixing, feature development, refactoring, and performance optimization.
• Build deterministic verification systems and golden reference solutions.
• Evaluate AI agents' ability to reason through complex codebases and use MCP tools effectively.
• Develop realistic, reproducible software engineering scenarios.
• Document solutions and provide clear technical feedback.

Skills

Python 3

Java

Rust

C++

TypeScript

algorithms & data structures

bug fixing

feature implementation

codebase refactoring

performance optimization

communication

detail oriented

Job description

YO AI Labs seeks experienced Senior Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using MCP tools.

You will design reproducible environments, deterministic verification, and reference solutions for tasks such as bug fixing, feature implementation, codebase refactoring, and performance optimization. No prior AI experience is required.