Remote Code Review Engineer — Evaluation Specialist

AI Trainer Jobs; United States; Remote; USD 158,000 - 200,000; Full time

Job summary

AI Trainer Jobs is seeking a Software Engineering Evaluation Specialist for a remote contract role. You will evaluate production code, reproduce failures, and write tests the model should have written to guide fixes.

You will judge generated code and outputs, read debugging traces, and provide a clear rationale tied to rubric; independence and a 10-hour-per-week availability are expected.

Qualifications

• Strong day-job engineering experience — you can read, run, and debug unfamiliar code for Software Engineering Evaluation Specialist work.
• Comfort writing concise unit tests that capture a single failure mode.
• Familiarity with a testing framework. pytest, Jest, or JUnit. Go test, RSpec, or whatever you use.
• Clear written reasoning — your review note has to convince another senior engineer.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Run and reproduce candidate code outputs in a sandboxed environment for Software Engineering Evaluation Specialist assignments.
• Grade software engineering evaluation engineering review solutions for correctness, style, and edge-case handling.
• Write minimal failing tests that demonstrate the bug a model output missed.
• Compare paired solutions and rank them with a written rationale tied to the rubric.

Skills

Code review

Debugging

Unit testing

Software engineering judgment

Software Engineering Evaluation

Tools

pytest

Jest

JUnit

Go test

RSpec

Job description

AI Trainer Jobs is seeking a Software Engineering Evaluation Specialist for a remote contract role. You will evaluate production code, reproduce failures, and write tests the model should have written to guide fixes.

You will judge generated code and outputs, read debugging traces, and provide a clear rationale tied to rubric; independence and a 10-hour-per-week availability are expected.