Lead AI Engineer - Agentic Systems & Evals | Klaviyo NEW

The Briefed; Boston (MA); In loco; USD 176.000 - 264.000; Tempo pieno

Vantaggi offerti da questo lavoro

Health insurance

Equity grants

Wellness stipend

Parental leave

Descrizione del lavoro

Klaviyo seeks a Lead Engineer, Applied AI (Customer Agent) to drive technical and product direction for a core area of the Customer Agent platform. You will work hands-on on orchestration, guardrails, context/memory, and failure handling while elevating engineering patterns across the org.

You will partner with Product, Design, Data Science, and GTM leaders to shape roadmaps, mentor engineers, and evaluate emerging AI tech for production use.

Competenze

• Production agentic experience in building, scaling and supporting live systems.
• Operational security against abuse and maintaining reliability under high traffic.
• Ability to own technical and product direction for a significant domain.

Mansioni

• Set the technical vision and product direction from architecture to production launch.
• Write production code, review designs, and resolve complex challenges in orchestration and failure handling.
• Define reusable engineering patterns, standards, and safety frameworks for the team.

Conoscenze

Agentic systems

Reliability & security

Technical leadership

Hands-on coding

Communication

Strumenti

LLM frameworks

Agentic tools

AI observability

Descrizione del lavoro

As Lead Engineer, Applied AI (Customer Agent), you will set the technical and product direction for a core area of Klaviyo's Customer Agent platform end-to-end. The Customer Agent team is building a conversational AI platform that lets brands serve their own customers at scale. Moving beyond prototypes and demos, this team ships against live customer traffic, measures quality with rigorous evaluation frameworks, and treats reliability as a foundational product feature.

In this role, you will lead 0-to-1 agentic product development with the speed and ownership of a startup- backed by the scale and resources of Klaviyo. You will stay fully hands-on with code, solving the hardest non-deterministic system challenges spanning agent orchestration, guardrails, context/memory, and failure handling while up-leveling engineering patterns across the organization.

Key Responsibilities

Agentic Architecture & Technical Leadership

• Technical & Product Ownership: Set the technical vision and product direction for a core domain within Customer Agent—from architecture through production launch and post-launch scaling.
• Hands-On Problem Solving: Personally write code, review architecture designs, and resolve the most complex technical challenges: orchestration, non-deterministic failure handling, evals, guardrails, and context/memory management.
• Architectural Guardrails & Patterns: Define reusable engineering patterns, standards, and safety frameworks adopted by other engineers across the Applied AI organization.
• Cross-Functional Partnership: Partner directly with Product Management, Design, Data Science, and Go-To-Market leaders to shape product roadmaps and strategic feature bets.
• Technical Mentorship: Mentor and up-level engineers through architectural design reviews, technical pairing, and leading by example rather than direct people management.
• Emerging AI Technology Evaluation: Evaluate cutting-edge AI research, LLM frameworks, and agentic tools, bringing proven, production-ready technologies into Klaviyo's stack.

Requirements

Production Agentic Systems & Engineering Depth

• Production Agentic Experience: Demonstrated track record of building, scaling, and supporting production agentic systems (not just prototypes)—including real orchestration, evaluation frameworks, guardrails, and non-deterministic failure handling through live launches and operational incidents.
• Production Reliability & Security: Hard-learned operational experience securing agentic systems against abuse/misuse and keeping non-deterministic LLM products reliable under growing traffic.
• Technical Leadership & Scope: Proven experience owning technical and product direction for a significant domain, shipping architecture that other engineers built upon.
• Hands-On Coding & Execution: Active contributor who writes production code, reviews architecture decisions, and debugs production issues directly.
• Communication & Influence: Ability to translate complex technical trade-offs clearly to engineering and product audiences, framing ambiguous problems into actionable technical plans.

Preferred Qualifications

• Experience building or scaling a multi-tenant agent platform where external customers configure unique agent behaviors on shared infrastructure.
• Track record of launching 0-to-1 products or working within fast-paced startup environments.
• Direct experience with production AI evaluation systems, AI observability, or human-in-the-loop workflows for LLM-powered applications.

Total Rewards & Benefits

• Compensation & Equity: US base pay range ($176,000 - $264,000 USD) + eligibility for annual cash bonus + company equity grants.
• Health & Wellness: Comprehensive healthcare, dental, and vision insurance packages, wellness stipends, and mental health support.
• Culture & Growth: Inclusive workspace, generous paid time off, parental leave, and professional growth opportunities.