Lead AI Quality Engineer — Platform & Observability

Circle; Faro; Remote; EUR 125.000 - 178.000; Vollzeit

Zusammenfassung

Circle is hiring a Lead Engineer for its AI Quality engineering team. You will build evaluation frameworks, observability tooling, and diagnostic infrastructure to ensure Circle's AI agents perform reliably and efficiently.

This hands-on player-coach role requires leading and growing the team while remaining actively coding and shaping the technical roadmap. You will work closely with AI Core engineers to drive changes that improve quality, latency, and cost, in a fast-paced, remote-friendly

Qualifikationen

• 7+ years building and shipping production software, incl. LLM-powered agents.
• Proficiency with Ruby on Rails and Python; willing to learn quickly.
• Experience building eval/infrastructure for ML/AI systems and tooling.
• Ability to design datasets and annotation workflows for evaluation.

Aufgaben

• Build and own evaluation infrastructure, pipelines, scorers, and datasets.
• Diagnose quality breakdowns across agent pipelines and tool usage.
• Grow annotation workflows and AI-generated or simulated conversations.
• Run experiments across prompts, models, and agent harness; optimize costs.
• Lead and grow the AI Quality engineering team; stay hands-on in code.
• Collaborate with AI Core engineering to improve product quality.

Kenntnisse

LLM-powered agents

Ruby on Rails / Python

Observability / eval infra

Experimentation mindset

English proficiency (CEFR C2)

Tools

Braintrust

LangSmith

CI/CD pipelines

Jobbeschreibung

Circle is hiring a Lead Engineer for its AI Quality engineering team. You will build evaluation frameworks, observability tooling, and diagnostic infrastructure to ensure Circle's AI agents perform reliably and efficiently.

This hands-on player-coach role requires leading and growing the team while remaining actively coding and shaping the technical roadmap. You will work closely with AI Core engineers to drive changes that improve quality, latency, and cost, in a fast-paced, remote-friendly