Software Engineers

Great Value Hiring; Brasil; On-site; BRL 247,000 - 352,000; Full time

Job summary

Great Value Hiring is seeking Software Engineers to participate in coding tasks for AI evaluation. The study tests the effectiveness of coding tasks used to evaluate AI agents and collects feedback on task realism.

Participants review environments, verify evaluation harness logic, and discuss potential flaws. The role involves code review, technical discussion, and providing actionable feedback on task structure.

Qualifications

• Professional experience as a software engineer
• Located in Brazil or Argentina
• Familiarity with building or verifying coding tasks and test environments
• Comfortable discussing technical architectures and evaluation frameworks

Responsibilities

• Review realistic programming tasks designed for AI evaluation
• Assess the accuracy and structure of various evaluation harnesses
• Walk us through your thought process while checking code validity
• Provide actionable feedback on how to improve task complexity and realism

Job description

Software Engineers: Coding Tasks for AI Evaluation [$40 per hour]

A paid study on the effectiveness of coding tasks used to evaluate AI agents.

How It Works

The candidate will review a series of proposed coding tasks and assess their suitability for testing AI agents and verify the logic of the evaluation harnesses and provide feedback on their realistic application. The candidate will walk through the environments and point out potential flaws or improvements. The session involves a mix of code review, technical discussion, and direct feedback on the task structures.

Who This Is For

Software engineers based in Brazil and Argentina with strong technical backgroundsCandidates having direct experience building, reviewing, or testing evaluation harnesses and programming tasksBackend developers, full-stack engineers, and quality assurance automation specialists.

Role Responsibilities

• Review realistic programming tasks designed for AI evaluation
• Assess the accuracy and structure of various evaluation harnesses
• Walk us through your thought process while checking code validity
• Provide actionable feedback on how to improve task complexity and realism

Good Candidature

• Professional experience as a software engineer or developer
• Located in Brazil or Argentina
• Familiarity with building or verifying coding tasks and test environments
• Comfortable discussing technical architectures and evaluation frameworks