Remote AI Software Benchmark Engineer

OpenTrain AI, Inc.; United States; Remote; USD 83,000 - 165,000; Part time

Job summary

OpenTrain AI, Inc. is seeking a remote expert software engineer to create realistic AI evaluation tasks, write precise instructions, and establish clear evaluation standards.

You will also build reproducible environments, reference solutions, and tests, and review AI-generated code for quality and robustness. This fully remote, part-time contractor role pays $60–$120 per hour equivalent, paid per accepted task that meets project specifications, with less than 20 hours per week.

Qualifications

• Expert-level software engineering ability with prior paid experience training, evaluating, benchmarking, or improving AI systems.
• Mid-Senior level experience expected.
• Experience building, evaluating, and benchmarking AI-related software tasks.

Responsibilities

• Create demanding software engineering tasks for AI model evaluation.
• Write task instructions and define objective standards for correct solutions.
• Build reproducible environments, reference solutions, tests, and evaluation criteria.
• Review generated code for correctness, robustness, and reproducibility.
• Identify problems in implementation, debugging, dependencies, and performance.
• Contribute technical judgment to coding evaluations and benchmark-style tasks.

Skills

Software design

Debugging

Testing

Version control

Dependency management

Tools

Docker

CI/CD

Command-line tools

Sandboxed environments

Linux

Job description

OpenTrain AI, Inc. is seeking a remote expert software engineer to create realistic AI evaluation tasks, write precise instructions, and establish clear evaluation standards.

You will also build reproducible environments, reference solutions, and tests, and review AI-generated code for quality and robustness. This fully remote, part-time contractor role pays $60–$120 per hour equivalent, paid per accepted task that meets project specifications, with less than 20 hours per week.