Remote AI Platform Engineering Lead – Quality & Observability

Lever, Inc.; Netherlands; Remote; EUR 133,000 - 169,000; Full time

Benefits offered by this job

Equity

35 days PTO

Fully remote

Company retreats

Health & wellbeing

Professional development

Global team

Job summary

Lever, Inc. is seeking a Lead Engineer, AI Platform based in Netherlands.

You will lead the engineering foundation behind reliable, measurable, and scalable AI-powered features, building evaluation frameworks, observability tooling, and diagnostic infrastructure for production AI agents. This role blends hands-on software development with technical leadership and people management, guiding experiments, cost and latency optimizations, and collaboration with AI Core teams to improve product

Qualifications

• 7+ years of production software experience, including AI-focused systems.
• Experience with complex, tool-using AI systems involving multiple tools and sub-agents.
• Proven track record shipping reliable software and measuring impact.

Responsibilities

• Design, build, and own evaluation infrastructure including CI/CD pipelines, scorers, datasets, and multi-turn AI agent assessment.
• Develop observability and diagnostic capabilities across planning, execution, tool selection, and agent trajectories.
• Investigate failures in complex AI workflows and turn findings into prototypes or priorities.
• Build datasets through human annotation, AI-generated samples, and simulated conversations to expand coverage.
• Develop structured experimentation frameworks for prompts, models, and agent harnesses across production baselines.
• Identify cost and latency optimization opportunities through model selection, caching, and routing.
• Set the technical direction and manage day-to-day priorities for the AI Quality team while hands-on.
• Partner with AI Core teams to measure changes and prove quality improvements.
• Establish engineering practices and evaluation approaches for reliable, scalable production AI systems.

Skills

Leadership

Python

Ruby on Rails

Observability

Evaluation pipelines

CI/CD

Data annotation workflows

English proficiency

Cloud/DevOps basics

Tools

Braintrust

LangSmith

Observability tooling

Job description

Lever, Inc. is seeking a Lead Engineer, AI Platform based in Netherlands.

You will lead the engineering foundation behind reliable, measurable, and scalable AI-powered features, building evaluation frameworks, observability tooling, and diagnostic infrastructure for production AI agents. This role blends hands-on software development with technical leadership and people management, guiding experiments, cost and latency optimizations, and collaboration with AI Core teams to improve product