Remote Software Engineer - Evaluation Harness Reviewer

AI Trainer Jobs; United States; Remote; USD 83,000 - 96,000; Part time

Job summary

AI Trainer Jobs seeks experienced software engineers to participate in a remote, paid study evaluating coding environments and evaluation harnesses used to benchmark AI agents. You will review tasks for realism, verify logic and test cases, and share your reasoning as you walk through complex code structures.

Independent contractor role with weekly payments and flexible scheduling. Join a team that values rigorous code review and practical feedback on building robust evaluation harnesses for AI

Qualifications

• Active professional experience as a software engineer.
• Hands-on experience building or maintaining test suites and evaluation harnesses.
• Familiarity with complex real-world software architecture.

Responsibilities

• Review realistic programming tasks for accuracy and difficulty.
• Verify the logic and test cases within provided evaluation harnesses.
• Assess whether coding environments effectively measure software engineering skills.
• Walk us through your thought process while analyzing complex code structures.

Skills

Software engineering

Test suites

Evaluation harnesses

Code review

Tools

Test frameworks

Job description

AI Trainer Jobs seeks experienced software engineers to participate in a remote, paid study evaluating coding environments and evaluation harnesses used to benchmark AI agents. You will review tasks for realism, verify logic and test cases, and share your reasoning as you walk through complex code structures.

Independent contractor role with weekly payments and flexible scheduling. Join a team that values rigorous code review and practical feedback on building robust evaluation harnesses for AI