Freelance SWE-Bench AI Task Auditor (Remote)

Jobgether; South Africa; On-site; ZAR 1,542,000 - 2,202,000; Full time

Benefits offered by this job

Remote work

Contract-based freelance

Flexible schedule

Project-based work

Competitive hourly pay

Job summary

Jobgether in South Africa is seeking a SWE-Bench AI Task Auditor for a freelance AI trainer project. You will audit software engineering tasks used to train and evaluate AI models, focusing on realism, solvability, and reproducibility.

You will review codebases, validate tests, identify integration issues, and provide actionable feedback to improve task quality. This remote, project-based role requires deep software engineering expertise and the ability to work independently.

Qualifications

• Demonstrable professional experience in software engineering with real-world projects.
• Ability to evaluate tasks against practical development standards.
• Excellent written communication for precise technical feedback.

Responsibilities

• Evaluate software engineering tasks for realism, solvability, reproducibility, and SWE-Bench alignment.
• Review complex codebases and real-world scenarios to reflect practical challenges.
• Assess reliability of tests and evaluation criteria; identify gaps.
• Identify integration issues, logic errors, test failures, and inconsistencies affecting task quality.
• Troubleshoot technical scenarios and provide actionable feedback to improve tasks.
• Document findings clearly to enable efficient remediation.

Skills

Software engineering

Codebase navigation

AI evaluation

Analytical skills

Feedback writing

Tools

Python

Code review tools

Job description

Jobgether in South Africa is seeking a SWE-Bench AI Task Auditor for a freelance AI trainer project. You will audit software engineering tasks used to train and evaluate AI models, focusing on realism, solvability, and reproducibility.

You will review codebases, validate tests, identify integration issues, and provide actionable feedback to improve task quality. This remote, project-based role requires deep software engineering expertise and the ability to work independently.