AI Model Evaluation DevOps Engineer

Mercor; Helsinki; On-site; EUR 381,000 - 571,000; Full time

Job summary

Mercor in Helsinki seeks contributors to evaluate frontier AI coding models through structured technical assessments, focusing on realistic infrastructure engineering workflows and model evaluation.

You will use frontier AI coding agents to complete complex tasks, review cloud platforms, Kubernetes, CI/CD, observability, and automation, and identify bugs and edge cases.

The role is sprint-based, with compensation of $400 per accepted task; typical tasks take 2–3 hours, and ramp-up is required.

Qualifications

• 2+ years of professional DevOps, SRE, or Cloud Engineering experience.
• Experience with cloud platforms and Kubernetes.
• Familiarity with automation and CI/CD pipelines.
• Regular use of AI coding agents or related tooling.

Responsibilities

• Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
• Review model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, observability, and infrastructure automation.
• Identify bugs, edge cases, reliability issues, and failure modes.
• Compare outputs from multiple frontier models and assess their strengths and weaknesses.
• Apply professional engineering judgment to realistic infrastructure engineering scenarios.

Skills

DevOps

SRE

Cloud Engineering

Tools

AWS

Azure

GCP

Kubernetes

Terraform

CI/CD

Observability

Job description

Mercor in Helsinki seeks contributors to evaluate frontier AI coding models through structured technical assessments, focusing on realistic infrastructure engineering workflows and model evaluation.

You will use frontier AI coding agents to complete complex tasks, review cloud platforms, Kubernetes, CI/CD, observability, and automation, and identify bugs and edge cases.

The role is sprint-based, with compensation of $400 per accepted task; typical tasks take 2–3 hours, and ramp-up is required.