Freelance SWE-Bench AI Task Auditor (Remote)

Jobgether; Ireland; On-site; EUR 83,000 - 119,000; Full time

Benefits offered by this job

Remote work environment

Freelance contract structure for flex

Contractor to supply own equipment and

Job summary

Jobgether partners with a client to recruit a SWE-Bench AI Task Auditor – Freelance AI Trainer project based in Ireland. The role involves auditing tasks used to train and evaluate AI models, focusing on realism, solvability, and reproducibility.

This remote freelance opportunity is designed for experienced software engineers who can work independently to review complex codebases, identify issues, and provide actionable feedback to improve task quality and alignment with SWE-Bench standards.

Qualifications

• Demonstrable deep software engineering experience with real-world development.
• Ability to audit tasks against SWE-Bench standards and real-world scenarios.
• Strong analytical and problem-solving skills for complex evaluation.

Responsibilities

• Audit and evaluate SWE-Bench tasks for realism, solvability, and reproducibility.
• Review codebases and real-world scenarios to verify accuracy of tasks.
• Identify issues in tests and criteria and provide actionable feedback.
• Document findings clearly to support task improvements.

Skills

Software engineering

Codebase navigation

SWE-Bench knowledge

Job description

Jobgether partners with a client to recruit a SWE-Bench AI Task Auditor – Freelance AI Trainer project based in Ireland. The role involves auditing tasks used to train and evaluate AI models, focusing on realism, solvability, and reproducibility.

This remote freelance opportunity is designed for experienced software engineers who can work independently to review complex codebases, identify issues, and provide actionable feedback to improve task quality and alignment with SWE-Bench standards.