Remote Lead Engineer - AI Quality & Evaluation Platform

CIRCLE; Lisboa; Teletrabalho; EUR 90 000 - 150 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

35 days PTO

Remote work

Company retreats

Resumo da oferta

Circle is seeking a Lead Engineer for AI Quality to build evaluation frameworks, observability tooling, and diagnostic infrastructure for its AI agents. This hands-on coach role also leads the AI Quality engineering team and collaborates with AI Core engineering to deliver reliable, scalable AI features.

You will design datasets, run experiments across prompts and models, and optimize cost and latency while maintaining high quality across Circle’s AI-powered tools.

Qualificações

• 7+ years of experience building and shipping production software with multi-tool agents.
• Proficiency in Ruby on Rails and Python.
• Experience building eval/inference infrastructure for ML/AI systems.
• Experience designing datasets, annotation workflows, or labeling pipelines for ML/AI evaluation.
• Learn fast, experiment aggressively, and thrive in a dynamic environment.
• Proficient in English at CEFR C2 / ILR 5.

Responsabilidades

• Build and own our evaluation infrastructure; design CI/CD pipelines, scorers, and datasets for agent evaluation.
• Diagnose where quality breaks down across the agent pipeline and prioritize fixes.
• Grow the datasets with AI-generated and simulated conversations.
• Run structured experiments across prompts, models, and the agent harness; optimize cost and latency.
• Lead and grow the AI Quality engineering team and set technical direction.
• Partner with AI Core engineering to ensure changes improve quality, not just shipping.

Conhecimentos

Ruby on Rails

Python

LLM-powered agents

Observability

Dataset design

Experimentation

English proficiency

Ferramentas

Braintrust

LangSmith

Descrição da oferta de emprego

Circle is seeking a Lead Engineer for AI Quality to build evaluation frameworks, observability tooling, and diagnostic infrastructure for its AI agents. This hands-on coach role also leads the AI Quality engineering team and collaborates with AI Core engineering to deliver reliable, scalable AI features.

You will design datasets, run experiments across prompts and models, and optimize cost and latency while maintaining high quality across Circle’s AI-powered tools.