Remote Lead Engineer, AI Quality Platform

Circle; United States; Remote; USD 180,000 - 240,000; Full time

Benefits offered by this job

Remote, Anywhere

US benchmarks compensation globally

Equity with refresh grants

35 days PTO

Job summary

Circle is seeking a Lead Engineer for the AI Quality engineering team to build evaluation frameworks, observability tooling, and diagnostic infrastructure for our AI agents. You will lead and mentor the team while staying hands-on in code and collaborating with AI Core and product teams.

This role requires 7+ years shipping production software with ML/AI systems, strong Ruby on Rails and Python skills, and a track record of designing eval pipelines and dashboards.

Qualifications

• 7+ years shipping production software with LLM-powered agents.
• Experience with tool-using systems, orchestration, and sub-agents.
• Proficient in Ruby on Rails and Python.
• Experience building evaluation or observability infra for ML/AI.
• Built eval pipelines, dashboards, or CI/CD for evals.

Responsibilities

• Build and own evaluation infrastructure.
• Design CI/CD pipelines, scorers, and datasets for AI agents.
• Diagnose where quality breaks down across the agent pipeline.
• Lead and grow the AI Quality engineering team.
• Set technical direction while coding hands-on.

Skills

Production software

LLM-powered agents

Ruby on Rails

Python

Observability infra

Eval pipelines

Experimentation

English proficiency

Tools

Braintrust

LangSmith

Job description

Circle is seeking a Lead Engineer for the AI Quality engineering team to build evaluation frameworks, observability tooling, and diagnostic infrastructure for our AI agents. You will lead and mentor the team while staying hands-on in code and collaborating with AI Core and product teams.

This role requires 7+ years shipping production software with ML/AI systems, strong Ruby on Rails and Python skills, and a track record of designing eval pipelines and dashboards.