Lead AI Quality Engineer - Remote

Circle; Portugal; Teletrabalho; EUR 160 000 - 213 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Equity grants

35 days PTO

Remote-only team

Resumo da oferta

Circle is seeking a Lead Engineer for the AI Quality team. You’ll build evaluation frameworks, observability tooling, and diagnostic infrastructure to ensure Circle's AI agents perform reliably, efficiently, and at scale.

This hands-on leader will guide a growing team and partner with AI Core engineering to drive real quality improvements across production systems. You’ll design datasets, annotation workflows, and experiments, train the team, and ship improvements that reduce latency and cost

Qualificações

• 7+ years of experience building and shipping production software with complex, tool-using systems.
• Experience with LLM-powered agents and multi-tool orchestration.
• Proficiency in Ruby on Rails and Python; strong data/evaluation focus.

Responsabilidades

• Build and own evaluation infrastructure, CI/CD pipelines, scorers, and datasets for AI agents.
• Diagnose quality breaks across the agent pipeline and propose prototypes or priorities.
• Grow annotation workflows and AI-generated or simulated conversations.
• Run experiments across prompts, models, and agent harness; evaluate models and costs.
• Lead and grow the AI Quality engineering team while contributing hands-on coding.
• Collaborate with AI Core engineering to ensure improvements in quality are real.

Conhecimentos

LLM-powered agents

Experimentation

Observability infrastructure

Ruby on Rails

Python

CI/CD

Data evaluation

Ferramentas

Braintrust

LangSmith

Descrição da oferta de emprego

Circle is seeking a Lead Engineer for the AI Quality team. You’ll build evaluation frameworks, observability tooling, and diagnostic infrastructure to ensure Circle's AI agents perform reliably, efficiently, and at scale.

This hands-on leader will guide a growing team and partner with AI Core engineering to drive real quality improvements across production systems. You’ll design datasets, annotation workflows, and experiments, train the team, and ship improvements that reduce latency and cost