Remote Freelance SWE-Bench AI Task Auditor

Jobgether; Netherlands; On-site; EUR 83,000 - 119,000; Full time

Benefits offered by this job

Remote work

Freelance contract

Job summary

Jobgether is seeking a SWE-Bench AI Task Auditor on a freelance basis, based in the Netherlands. You will audit advanced AI training tasks, focusing on realism, solvability, and reproducibility within real-world software engineering contexts.

You’ll review complex codebases, identify issues, and provide precise, actionable feedback to improve task quality and evaluation workflows while working independently on project-based assignments.

Qualifications

• Demonstrable professional experience in software engineering.
• Ability to evaluate tasks against practical SWE standards.
• Strong analytical and problem-solving skills.
• Excellent written communication for precise feedback.
• Ability to work independently on freelance projects.
• Security of a reliable internet connection.

Responsibilities

• Evaluate tasks for technical accuracy, realism, solvability, reproducibility, and SWE-Bench alignment.
• Review codebases and real-world scenarios to verify practical challenges.
• Assess reliability and completeness of tests and evaluation criteria.
• Identify integration issues, logic errors, and inconsistencies affecting task quality.
• Troubleshoot scenarios and provide actionable feedback to improve task implementations.
• Apply SWE expertise to meet high standards of rigor and relevance.
• Document findings clearly for efficient remediation.

Skills

Software engineering

Analytical skills

Problem solving

Independent work

Attention to detail

Written communication

Job description

Jobgether is seeking a SWE-Bench AI Task Auditor on a freelance basis, based in the Netherlands. You will audit advanced AI training tasks, focusing on realism, solvability, and reproducibility within real-world software engineering contexts.

You’ll review complex codebases, identify issues, and provide precise, actionable feedback to improve task quality and evaluation workflows while working independently on project-based assignments.