Remote DevOps & SRE Evaluation Specialist — Code Quality Evaluator

AI Trainer Jobs; United States; Remote; USD 83,000 - 124,000; Part time

Job summary

AI Trainer Jobs is seeking a DevOps and SRE Evaluation Specialist for remote, contractor work. You will reproduce candidate code outputs in a sandbox, grade cloud engineering solutions for correctness, and write minimal failing tests to expose edge-case bugs.

You will rank paired solutions with clear, rubric-based rationales for senior engineers. Ideal candidates have strong engineering experience, can read and run unfamiliar code, and maintain reliable async availability for at least 10 hours

Qualifications

• Strong day-job engineering experience; read, run, and debug unfamiliar code for DevOps/SRE evaluation work.
• Comfort writing concise unit tests that capture a single failure mode.
• Familiarity with a testing framework (pytest, Jest, JUnit, Go test, RSpec).
• Clear written reasoning—reviews must convince senior engineers.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Run and reproduce candidate code outputs in a sandboxed environment for DevOps and SRE Evaluation Specialist assignments.
• Grade cloud & platform engineering solutions for correctness, style, and edge-case handling.
• Write minimal failing tests that demonstrate the bug a model output missed.
• Compare paired solutions and rank them with a written rationale tied to the rubric.

Skills

Code review

Debugging

Unit testing

Software engineering judgment

Cloud & platform engineering

Tools

pytest

Jest

JUnit

Go test

RSpec

Job description

AI Trainer Jobs is seeking a DevOps and SRE Evaluation Specialist for remote, contractor work. You will reproduce candidate code outputs in a sandbox, grade cloud engineering solutions for correctness, and write minimal failing tests to expose edge-case bugs.

You will rank paired solutions with clear, rubric-based rationales for senior engineers. Ideal candidates have strong engineering experience, can read and run unfamiliar code, and maintain reliable async availability for at least 10 hours