Frontier AI DevOps Engineer & Model Evaluator

Mercor; Dublin; On-site; EUR 393,000 - 560,000; Part time

Job summary

Mercor is seeking contributors to evaluate frontier AI coding models through structured assessments focused on infrastructure engineering workflows.

You will review model outputs across cloud platforms, Kubernetes, CI/CD, observability, and automation, identifying reliability issues and failure modes. The role emphasizes professional engineering judgment and production-scale evaluation.

Qualifications

• 2+ years of professional DevOps, SRE, or Cloud Engineering experience.
• Experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, or observability tooling.
• Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
• Ability to evaluate model-generated infrastructure and reliability engineering solutions.
• Experience supporting production-scale systems is preferred.

Responsibilities

• Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
• Review model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, observability, and infrastructure automation.
• Identify bugs, edge cases, reliability issues, and failure modes.
• Compare outputs from multiple frontier models and assess their strengths and weaknesses.
• Apply professional engineering judgment to realistic infrastructure engineering scenarios.

Skills

DevOps/SRE

Cloud engineering

AI coding agents usage

Problem solving

Production systems experience

Tools

Kubernetes

Terraform

CI/CD pipelines

Observability tooling

Job description

Mercor is seeking contributors to evaluate frontier AI coding models through structured assessments focused on infrastructure engineering workflows.

You will review model outputs across cloud platforms, Kubernetes, CI/CD, observability, and automation, identifying reliability issues and failure modes. The role emphasizes professional engineering judgment and production-scale evaluation.