Remote Senior AI Code Evaluation Engineer

YO AI Labs; Calgary; Remote; CAD 90,000 - 131,000; Full time

Job summary

YO AI Labs is seeking experienced Senior Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using MCP tools.

You will design reproducible environments, deterministic verification, and reference solutions for tasks such as bug fixing, feature implementation, codebase refactoring, and performance optimization. No prior AI experience is required.

Qualifications

• Experience designing reinforcement learning environments for software tasks.
• Proficient in Python, Java, Rust, C++, or TypeScript.
• Strong knowledge of algorithms and data structures.
• Experience debugging complex software issues.
• Experience with codebase refactoring and feature implementation.
• Ability to optimize performance and tune systems.
• Excellent written and verbal communication.
• Detail-oriented with strong problem-solving.

Responsibilities

• Create reinforcement learning environments for software engineering tasks.
• Design tasks involving bug fixing, feature development, refactoring, and performance optimization.
• Build deterministic verification systems and golden reference solutions.
• Evaluate AI agents' ability to reason through complex codebases and use MCP tools effectively.
• Develop realistic, reproducible software engineering scenarios.
• Ensure tasks accurately measure coding ability, problem-solving, and tool usage.
• Document solutions and provide clear technical feedback.

Skills

Python 3

Java

Rust

C++

TypeScript

Algorithms and data structures

Bug fixing and debugging

Code refactoring and feature impl

Performance optimization

Communication skills

Attention to detail

Job description

YO AI Labs is seeking experienced Senior Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using MCP tools.

You will design reproducible environments, deterministic verification, and reference solutions for tasks such as bug fixing, feature implementation, codebase refactoring, and performance optimization. No prior AI experience is required.