Remote AI Task Auditor for SWE-Bench (Freelance)

Jobgether; France; Sur place; EUR 83 000 - 119 000; Plein temps

Avantages offerts par ce poste

Remote work

Flexible project-based contract

Résumé du poste

Jobgether on behalf of a partner company seeks a SWE-Bench AI Task Auditor for a freelance AI trainer project based in France. You will audit tasks used to train AI models, focusing on realism, solvability, and reproducibility in software engineering scenarios.

This independent role requires deep SWE expertise, meticulous testing, and clear feedback. You’ll work remotely on project-based assignments, with compensation in USD per hour, and will ensure alignment with SWE-Bench standards.

Qualifications

• Demonstrable professional experience and deep technical expertise in software engineering, including navigating complex codebases and real-world application development.
• Strong familiarity with software engineering concepts and the ability to evaluate tasks against practical development standards, including SWE-Bench requirements.
• Excellent analytical and problem-solving skills with the ability to rigorously test, troubleshoot, and evaluate complex technical scenarios.
• Strong attention to detail and the ability to identify subtle logic, integration, testing, and reproducibility issues.
• Excellent written communication skills for providing precise, constructive, and actionable technical feedback.
• Ability to work independently on project-based freelance assignments.

Responsabilités

• Evaluate software engineering tasks for technical accuracy, realism, solvability, reproducibility, and alignment with SWE-Bench standards.
• Review codebases and real-world scenarios to ensure tasks reflect practical software engineering challenges.
• Assess the reliability and completeness of tests, evaluation criteria, and expected outcomes.
• Identify integration issues, logic errors, test failures, and other inconsistencies affecting task quality.
• Troubleshoot complex technical scenarios and provide actionable feedback to improve task implementation and evaluation workflows.
• Document findings clearly so issues can be addressed efficiently.

Description du poste

Jobgether on behalf of a partner company seeks a SWE-Bench AI Task Auditor for a freelance AI trainer project based in France. You will audit tasks used to train AI models, focusing on realism, solvability, and reproducibility in software engineering scenarios.

This independent role requires deep SWE expertise, meticulous testing, and clear feedback. You’ll work remotely on project-based assignments, with compensation in USD per hour, and will ensure alignment with SWE-Bench standards.