AI-Driven DevOps Evaluator for Frontier Coding Agents

Mercor; New York (NY); On-site; USD 441,000 - 661,000; Full time

Job summary

Mercor partners with a leading AI research lab to support a Frontier Code Agents project. Contributors help evaluate and improve frontier AI coding models through structured technical assessments.

The work focuses on realistic infrastructure engineering workflows and model evaluation, reviewing model-generated implementations across cloud platforms, Kubernetes, CI/CD systems, observability, and infrastructure automation.

Qualifications

• : 2+ years of professional DevOps, SRE, or Cloud Engineering experience.
• : Experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, or observability tooling.
• : Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
• : Ability to evaluate model-generated infrastructure and reliability engineering solutions.
• : Experience supporting production-scale systems is preferred.

Responsibilities

• : Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
• : Review model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, observability, and infrastructure automation.
• : Identify bugs, edge cases, reliability issues, and failure modes.
• : Compare outputs from multiple frontier models and assess their strengths and weaknesses.
• : Apply professional engineering judgment to realistic infrastructure engineering scenarios.

Skills

DevOps

SRE

Cloud engineering

AWS

Azure

GCP

Kubernetes

Terraform

CI/CD pipelines

Observability tooling

Tools

Cursor

Claude Code

Codex

Windsurf

Gemini CLI

Job description

Mercor partners with a leading AI research lab to support a Frontier Code Agents project. Contributors help evaluate and improve frontier AI coding models through structured technical assessments.

The work focuses on realistic infrastructure engineering workflows and model evaluation, reviewing model-generated implementations across cloud platforms, Kubernetes, CI/CD systems, observability, and infrastructure automation.