Lead Engineer, AI Platform

Lever, Inc.; Portugal; Teletrabalho; EUR 136 000 - 166 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Fully remote environment

Equity & growth potential

Flexible schedule & autonomy

Global team & international travel

Resumo da oferta

Jobgether seeks a Lead Engineer, AI Platform for a Germany-based partner company. You will lead the engineering foundation behind scalable AI-powered features, building evaluation frameworks, observability tooling, and diagnostic infrastructure for production AI systems.

You'll combine hands-on software development with people leadership, shaping the AI Quality function while collaborating with AI Core teams. Strong English, fast-paced delivery, and ownership culture are essential.

Qualificações

• 7+ years of experience building and shipping production software, ideally with LLM-powered agents.
• Experience with complex tool-using AI systems involving planning, orchestration, or sub-agents.
• Ability to demonstrate shipped software and how its effectiveness was measured.
• Experience with Ruby on Rails and/or Python and quickly adopting new tech.
• Experience building evaluation/observability infra for ML/AI systems (pipelines, dashboards, CI/CD).
• Familiarity with evaluation frameworks such as Braintrust, LangSmith or similar tools.
• Experience designing datasets, annotation workflows, or labeling pipelines for ML/AI evaluation.
• Ability to learn quickly, experiment, and translate empirical results into decisions.
• Strong technical leadership and people-management capabilities.
• Excellent English proficiency spoken, written, and reading (CEFR C2).
• Alignment with collaborative, ownership-oriented engineering culture.

Responsabilidades

• Design, build, and own evaluation infrastructure and CI/CD pipelines for AI agents.
• Develop observability and diagnostic capabilities across tool calls and multi-turn conversations.
• Investigate failures and translate findings into prototypes or priorities for AI engineering teams.
• Expand datasets via human annotation, AI-generated examples, and simulated conversations.
• Create structured experimentation for prompts, models, and agent harnesses; evaluate new/open-source models.
• Identify opportunities to reduce AI system cost and latency through design choices.
• Lead AI Quality engineering team while remaining hands-on.
• Collaborate with AI Core teams to measure product changes and improve quality.
• Establish engineering practices and evaluation approaches for scalable production AI.

Conhecimentos

LLM-powered agents

Complex tool-using AI systems

Shipped software + metrics

Ruby on Rails

Python

Eval/Observability infra

Datasets & annotation workflows

Learning agility

Technical leadership

English proficiency (CEFR C2)

Collaborative culture

Ferramentas

Braintrust

LangSmith

Descrição da oferta de emprego

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Lead Engineer, AI Platform based in Germany.

This role offers the opportunity to lead the engineering foundation behind reliable, measurable, and scalable AI-powered features. You’ll build evaluation frameworks, observability tooling, and diagnostic infrastructure that reveal how AI agents perform in real production environments. The position combines hands-on software engineering with technical leadership and people management. You’ll investigate quality issues across complex agent workflows, develop datasets and evaluation systems, and run experiments across models, prompts, and agent architectures. You’ll also help optimize AI systems for cost, latency, reliability, and overall user experience. Working in a highly remote and asynchronous environment, you’ll collaborate closely with AI engineering teams while shaping the technical direction of a growing AI Quality function.

Accountabilities:

• Design, build, and own evaluation infrastructure, including CI/CD pipelines, scorers, datasets, and systems for assessing AI agents from individual tool calls through complete multi-turn conversations.
• Develop observability and diagnostic capabilities to identify exactly where quality issues occur across planning, execution, tool selection, and complex agent trajectories.
• Investigate failures across sophisticated AI workflows and turn findings into technical prototypes, improvements, or clearly defined priorities for AI engineering teams.
• Build and expand datasets through human annotation, AI-generated examples, and simulated conversations to increase evaluation coverage efficiently.
• Develop structured experimentation frameworks for prompts, models, and agent harnesses, including evaluation of new and open-source models against production baselines.
• Identify opportunities to improve AI system cost and latency through model selection, caching, routing, and other optimization strategies.
• Set the technical direction and manage day-to-day priorities for the AI Quality engineering team while remaining actively involved in hands-on development.
• Partner closely with AI Core engineering teams to ensure changes to AI products can be measured effectively and demonstrably improve quality.
• Establish engineering practices and evaluation approaches that support reliable, efficient, and scalable production AI systems.

Requirements:

• 7+ years of experience building and shipping production software, ideally including LLM-powered agents capable of taking real actions within products.
• Experience working with complex, tool-using AI systems involving multiple tools, planning, orchestration, or sub-agents rather than only simple, single-turn assistants.
• Strong ability to demonstrate shipped software and explain how its effectiveness and reliability were measured.
• Experience with Ruby on Rails and/or Python, with the ability to become productive quickly in technologies that may be new to you.
• Experience building evaluation or observability infrastructure for ML/AI systems, including evaluation pipelines, scorers, dashboards, or CI/CD systems for evaluations.
• Familiarity with evaluation frameworks such as Braintrust, LangSmith, or similar tools.
• Experience designing datasets, annotation workflows, or labeling pipelines for machine learning or AI evaluation.
• Ability to learn quickly, experiment extensively, and use empirical results to guide technical decisions.
• Comfortable operating in a fast-paced environment with ambiguity and changing technical requirements.
• Strong technical leadership and people-management capabilities, with the ability to balance team leadership and hands-on engineering.
• Excellent English proficiency in spoken, written, and reading communication, equivalent to CEFR C2 / ILR 5.
• Strong alignment with a collaborative, ownership-oriented engineering culture.

Benefits:

• Annual cash compensation of $170,000 USD, benchmarked to U.S. compensation levels regardless of location.
• Equity in the company, including ongoing refresh grants.
• 35 days of paid time off per year.
• Fully remote work environment.
• Significant flexibility and autonomy in how you organize your work.
• Twice-yearly company retreats in international destinations.
• Benefits supporting health, wellbeing, and professional development.
• Opportunity to lead and grow an AI Quality engineering team while remaining hands‑on technically.
• Exposure to advanced AI agents, evaluation infrastructure, observability, experimentation, and production AI optimization.
• Opportunity to work with a globally distributed team across multiple countries and time zones.

How Jobgether works:

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-filling candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Data Privacy Notice:

By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.