Software Engineer (Remote | $60–$120/hr)

Synthires; United States; Remote; AUD 121,000 - 241,000; Part time

Job summary

Synthires is seeking experienced Software Engineers with prior AI training or model-evaluation experience for remote, flexible engagement. The role focuses on designing challenging AI software engineering tasks, detailing instructions, environments, and evaluation criteria.

You will assess AI-generated code and provide structured feedback to improve AI coding capabilities, aligning with project standards. The engagement is fully remote worldwide, requiring 15+ hours per week with a flexible

Qualifications

• Prior paid professional experience training, evaluating, benchmarking, or improving AI systems.
• Strong practical proficiency in Python.
• Strong knowledge of software design, debugging, testing, version control, and dependency management.
• Ability to assess code quality, identify technical issues, and validate software solutions.

Responsibilities

• Create realistic and technically challenging software engineering tasks to assess advanced AI models.
• Develop clear task instructions, reproducible environments, reference solutions, tests, and evaluation criteria.
• Evaluate AI-generated code and technical deliverables for correctness, robustness, and completeness.
• Identify issues involving implementation, debugging, dependencies, performance, testing, and reproducibility.
• Assess software design decisions, code quality, and the reliability of proposed solutions.
• Establish objective standards for evaluating technical submissions.
• Provide structured feedback to support the improvement of AI coding capabilities.

Skills

Python

Code quality assessment

Software debugging

Testing

Version control

Dependency management

Tools

Linux

Docker

CI/CD

Command-line tools

Sandboxed environments

Job description

Compensation: $60–$120 per hour equivalent (paid per accepted task)

Location: Fully Remote (Worldwide)

Time Commitment: 15+ hours per week

About the Opportunity

We are seeking experienced Software Engineers with prior professional AI-training or model-evaluation experience to support projects focused on improving advanced AI systems.

In this role, you will apply your software engineering expertise to create challenging technical tasks, evaluate AI-generated code, identify implementation failures, and establish objective standards for correct solutions. This opportunity is intended for candidates who have already completed substantive AI-training, model-evaluation, or benchmarking work.

Key Responsibilities

• Create realistic and technically challenging software engineering tasks to assess advanced AI models.
• Develop clear task instructions, reproducible environments, reference solutions, tests, and evaluation criteria.
• Evaluate AI-generated code and technical deliverables for correctness, robustness, and completeness.
• Identify issues involving implementation, debugging, dependencies, performance, testing, and reproducibility.
• Assess software design decisions, code quality, and the reliability of proposed solutions.
• Establish objective standards for evaluating technical submissions.
• Provide structured feedback to support the improvement of AI coding capabilities.

Required Qualifications

• Prior paid professional experience training, evaluating, benchmarking, or improving AI systems.
• Strong practical proficiency in Python.
• Strong knowledge of software design, debugging, testing, version control, and dependency management.
• Ability to assess code quality, identify technical issues, and validate software solutions.

Preferred Qualifications

• Experience creating AI-training tasks, reference solutions, tests, graders, or benchmark problems.
• Experience reviewing technical submissions from other AI trainers.
• Codeforces rating of approximately 1900 or higher, or strong results in another programming competition.
• Familiarity with Linux, Docker, CI/CD, command-line tools, and sandboxed environments.
• Experience with long-horizon or agentic coding evaluations.
• Availability to begin immediately.

Compensation & Engagement

• Compensation: $60–$120 per hour equivalent, paid per accepted task.
• Work Arrangement: Fully remote, worldwide.
• Time Commitment: 15+ hours per week.
• Schedule: Flexible.