Remote AI Code Evaluation Engineer

YO AI Labs; Philadelphia (Philadelphia County); Remote; USD 83,000 - 138,000; Part time

Job summary

YO AI Labs is seeking a Senior Software Engineer to contribute as a remote contractor (~15 hours/week). You will build reinforcement learning environments and require no prior AI experience.

The role focuses on designing reproducible tasks, deterministic verification, and performance optimization across software engineering challenges, collaborating with cross-functional teams in a distributed setup.

Qualifications

• Experience creating reinforcement learning environments for software tasks.
• Design deterministic verification systems and reference solutions.
• Evaluate AI agents on complex codebases using MCP tools.

Responsibilities

• Create reinforcement learning environments for software engineering tasks.
• Design tasks involving bug fixing, feature development, refactoring, and performance optimization.
• Build deterministic verification systems and golden reference solutions.
• Evaluate AI agents' ability to reason through complex codebases and use MCP tools effectively.
• Develop realistic, reproducible software engineering scenarios.
• Ensure tasks accurately measure coding ability, problem-solving, and tool usage.
• Document solutions and provide clear technical feedback.

Skills

Python 3

Java

Rust

C++

TypeScript

Job description

YO AI Labs is seeking a Senior Software Engineer to contribute as a remote contractor (~15 hours/week). You will build reinforcement learning environments and require no prior AI experience.

The role focuses on designing reproducible tasks, deterministic verification, and performance optimization across software engineering challenges, collaborating with cross-functional teams in a distributed setup.