Remote AI Code Evaluator & DevOps Engineer

Turing; United States; Remote; USD 11,000 - 22,000; Part time

Benefits offered by this job

Fully remote

Freelance contractor role

Job summary

Turing is seeking experienced Software Engineers to review and rate AI-generated coding solutions for real-world software tasks. You will assess agent runs, analyze code correctness, efficiency, safety, and overall quality, and provide structured feedback to improve AI-assisted software development.

You will read each agent’s full run, compare performances, and deliver evidence-based rationales and rankings to inform AI training initiatives.

Qualifications

• Minimum 3+ years of overall experience.
• Proficiency with at least one of the listed languages (Python, JS/TS, Java, Go, PHP).
• Familiarity with shell, Git, Docker, and CI/CD pipelines.
• Comfortable running and testing real-world projects locally.

Responsibilities

• Review three AI coding agent runs for the same developer task.
• Rate each run on 10 quality areas and assign an overall 1–5 score.
• Rank the three runs and provide written rationales with evidence.

Skills

Python

JavaScript/TypeScript

Java

Go

PHP

Tools

Shell commands

Git

Docker

Job description

Turing is seeking experienced Software Engineers to review and rate AI-generated coding solutions for real-world software tasks. You will assess agent runs, analyze code correctness, efficiency, safety, and overall quality, and provide structured feedback to improve AI-assisted software development.

You will read each agent’s full run, compare performances, and deliver evidence-based rationales and rankings to inform AI training initiatives.