Remote Freelance AI Task Auditor - SWE-Bench Expert

Jobgether; Turkey; On-site; TRY 4,674,000 - 6,678,000; Full time

Benefits offered by this job

Remote work

Flexible freelance contract

Contractor provides own computer &

Secure internet connection

Job summary

Jobgether is seeking a SWE-Bench AI Task Auditor for a freelance role based in Turkey. You will audit software engineering tasks used to train and evaluate AI models, ensuring they reflect realistic and technically rigorous development scenarios.

Work includes reviewing complex codebases, validating task logic, and identifying integration problems, test failures, and inconsistencies. The project emphasizes independent, project-based freelance work with flexible hours and remote collaboration.

Qualifications

• Deep technical expertise in software engineering with real-world codebases.
• Ability to evaluate tasks against SWE-Bench requirements.
• Strong analytical, problem-solving, and debugging skills.
• Excellent written feedback and communication.

Responsibilities

• Evaluate tasks for technical accuracy, realism, solvability, reproducibility.
• Review codebases and scenarios to reflect practical SWE challenges.
• Assess reliability and completeness of tests and criteria.
• Identify integration issues, logic errors, and test failures.
• Provide actionable feedback to improve task implementation.
• Document findings clearly for efficient remediation.
• Apply SWE expertise to ensure rigor and relevance.

Skills

Software engineering expertise

Codebase analysis

SWE-Bench knowledge

Technical feedback

Independent work

Job description

Jobgether is seeking a SWE-Bench AI Task Auditor for a freelance role based in Turkey. You will audit software engineering tasks used to train and evaluate AI models, ensuring they reflect realistic and technically rigorous development scenarios.

Work includes reviewing complex codebases, validating task logic, and identifying integration problems, test failures, and inconsistencies. The project emphasizes independent, project-based freelance work with flexible hours and remote collaboration.