Remote AI Software Evaluator & Benchmark Engineer

Synthires; San Francisco (CA); Remote; USD 83,000 - 165,000; Part time

Job summary

Synthires is seeking experienced Software Engineers to support AI-training and model-evaluation projects. This fully remote, worldwide opportunity requires 15+ hours per week and pays $60–$120 per hour equivalent for accepted tasks.

Ideal candidates have prior professional AI-training or model-evaluation experience and are proficient in Python, with strong skills in software design, debugging, testing, and version control. Flexible scheduling encouraged.

Qualifications

• Prior paid professional experience training, evaluating, benchmarking, or improving AI systems.
• Strong practical proficiency in Python.
• Strong knowledge of software design, debugging, testing, version control, and dependency management.
• Ability to assess code quality, identify technical issues, and validate software solutions.

Responsibilities

• Create realistic and technically challenging software engineering tasks to assess advanced AI models.
• Develop clear task instructions, reproducible environments, reference solutions, tests, and evaluation criteria.
• Evaluate AI-generated code and technical deliverables for correctness, robustness, and completeness.
• Identify issues involving implementation, debugging, dependencies, performance, testing, and reproducibility.
• Assess software design decisions, code quality, and the reliability of proposed solutions.
• Establish objective standards for evaluating technical submissions.
• Provide structured feedback to support the improvement of AI coding capabilities.

Skills

Python

Code analysis

Version control

Tools

Linux

Docker

CI/CD

Sandboxed environments

Job description

Synthires is seeking experienced Software Engineers to support AI-training and model-evaluation projects. This fully remote, worldwide opportunity requires 15+ hours per week and pays $60–$120 per hour equivalent for accepted tasks.

Ideal candidates have prior professional AI-training or model-evaluation experience and are proficient in Python, with strong skills in software design, debugging, testing, and version control. Flexible scheduling encouraged.