Lead AI Platform Engineer: Quality & Observability

Circle

Ireland

Remote

EUR 100,000 - 160,000

Full time

6 days ago

Be an early applicant

Application generator

A complete application in a minute — tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

35 days paid time off

Remote-first team

Equity and refresh grants

US benchmark compensation

Job summary

Circle is seeking a Lead Engineer for AI Quality to design and implement evaluation frameworks, observability tooling, and diagnostic infrastructure for AI-powered features. You will lead a growing team, own the evaluation stack, and work closely with AI Core to ensure reliability and efficiency.

This hands-on, player-coach role requires expertise in RoR/Python, evaluation pipelines, and creating scalable data-driven systems. The position is remote-first with a global team and generous benefits.

Qualifications

7+ years of experience building and shipping production software with LLM-powered agents.
Experience with multi-tool systems, planning or orchestration, sub-agents.
Strong Ruby on Rails proficiency; Python is a plus for data/eval work.
Experience building eval pipelines, dashboards, or CI/CD for evals.

Responsibilities

Build and own evaluation infrastructure, pipelines, scorers, and data for AI agents.
Diagnose quality breakdowns across the agent pipeline and prioritize improvements.
Grow annotation workflows and AI-generated data for faster product coverage.
Run experiments across prompts, models, and agent harness; optimize for cost/latency.
Lead and grow the AI Quality engineering team while remaining hands-on.
Partner with AI Core engineering to ensure changes improve quality, not just shipping.