Remote SWE-Bench AI Task Auditor | Freelance Evaluator

Jobgether; Belgique; Sur place; EUR 83 000 - 119 000; Plein temps

Avantages offerts par ce poste

Remote work

Flexible freelance contract

Competitive hourly rate

Résumé du poste

Jobgether is seeking a SWE-Bench AI Task Auditor for a freelance AI trainer project based in Belgium. You will audit tasks used to train and assess AI models, focusing on realism, solvability, and rigorous development standards.

You will review complex codebases, evaluate tests and criteria, identify integration issues, and provide detailed feedback to improve task quality while working independently in a flexible freelance setup.

Qualifications

• Professional experience in software engineering.
• Familiarity with SWE-Bench standards and evaluation goals.
• Ability to audit tasks for realism, solvability, reproducibility, and quality.

Responsabilités

• Evaluate software engineering tasks for technical accuracy, realism, solvability, reproducibility, and alignment with SWE-Bench standards.
• Review complex codebases and real-world application scenarios to determine whether tasks reflect practical software engineering challenges.
• Assess the reliability and completeness of tests, evaluation criteria, and expected outcomes of tasks.
• Identify codebase integration issues, logic errors, test failures, and other inconsistencies affecting task quality or evaluation accuracy.
• Troubleshoot complex technical scenarios and provide actionable feedback to improve task implementation and evaluation workflows.
• Apply specialized software engineering knowledge to ensure AI training and evaluation tasks meet high standards of technical rigor.
• Document findings clearly to enable efficient addressing of issues.

Connaissances

Software engineering

Codebase navigation

Test & troubleshooting

Analytical thinking

Written communication

Independent work

Description du poste

Jobgether is seeking a SWE-Bench AI Task Auditor for a freelance AI trainer project based in Belgium. You will audit tasks used to train and assess AI models, focusing on realism, solvability, and rigorous development standards.

You will review complex codebases, evaluate tests and criteria, identify integration issues, and provide detailed feedback to improve task quality while working independently in a flexible freelance setup.