Remote Lead Engineer, AI Quality & Evaluation

Circle Internet Financial; Lisboa; Remote; EUR 107,000 - 161,000; Full time

Benefits offered by this job

Equity in the company

35 days paid time off

Job summary

Circle Internet Financial is hiring a Lead Engineer for the AI Quality engineering team. You will build evaluation frameworks, observability tooling, and diagnostic infrastructure to ensure AI agents perform reliably and cost-efficiently. This is a hands-on, player-coach role with leadership responsibilities.

You’ll lead the team, set technical direction, and collaborate with AI Core engineering to drive improvements across the platform.

Qualifications

• 7+ years of experience building and shipping production software.
• Experience with Ruby on Rails or Python.
• Experience building evaluation or observability infrastructure for ML/AI systems.
• Experience designing datasets, annotation workflows, or labeling pipelines for ML/AI evaluation.

Responsibilities

• Build and own our evaluation infrastructure.
• Diagnose exactly where quality breaks down.
• Grow the datasets that make evaluation possible.
• Run structured experiments across prompts, models, and the agent harness.
• Lead and grow the AI Quality engineering team.
• Partner closely with AI Core engineering.

Skills

LLM-powered agents

Experimentation mindset

Observability tooling

Team leadership

Strong communication

Tools

Ruby on Rails

Python

Braintrust

LangSmith

Job description

Circle Internet Financial is hiring a Lead Engineer for the AI Quality engineering team. You will build evaluation frameworks, observability tooling, and diagnostic infrastructure to ensure AI agents perform reliably and cost-efficiently. This is a hands-on, player-coach role with leadership responsibilities.

You’ll lead the team, set technical direction, and collaborate with AI Core engineering to drive improvements across the platform.