Remote DevOps Engineer for AI Model Evaluation

Mercor

Ireland

On-site

EUR 86,000 - 117,000

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

Mercor is seeking a DevOps / SRE / Cloud Engineer (Coding Agent Experience) for a remote contract role. The job focuses on evaluating frontier AI coding agents to perform complex infrastructure tasks across cloud platforms including AWS/Azure/GCP, Kubernetes, and CI/CD pipelines.

You will review model-generated implementations and ensure reliability and scalability. Requirements include 2+ years in DevOps/SRE or Cloud Engineering, experience with cloud providers and Kubernetes, and familiarity

Qualifications

2+ years of professional DevOps, SRE, or Cloud Engineering experience.
Experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, or observability tooling.
Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
Ability to evaluate model-generated infrastructure and reliability engineering solutions.

Responsibilities

Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
Review model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, observability, and infrastructure automation.
Identify bugs, edge cases, reliability issues, and failure modes in model outputs.
Compare outputs from multiple frontier models to assess strengths and weaknesses.
Apply professional engineering judgment to realistic infrastructure engineering scenarios.