Lead AI Product Engineer - AI Agents & Emerging Products

System1; Los Angeles (CA); On-site; USD 180,000 - 270,000; Full time

Benefits offered by this job

Competitive salary + bonus + equity

Generous PTO + 11 company holidays

Open sick time

100% covered Medical, Dental, Vision

401k with match

Health & Dependent Care Flex Spending Account

Paid professional development

Leadership & growth opportunities

Virtual company and team building events

Job summary

A dynamic tech company located in Los Angeles is seeking a Lead AI Product Engineer to design and build cutting-edge AI agent systems. You will be responsible for multi-agent workflows that deliver real consumer value. The ideal candidate has experience with LLM-powered systems, strong backend engineering skills in Python or Node.js, and the ability to rapidly iterate based on user feedback. This position offers a competitive salary plus bonuses and equity, generous PTO, and comprehensive health benefits.

Qualifications

• Experience with LLM-powered systems in production.
• Strong backend engineering skills, ideally with Python and/or Node.js.
• Ability to design multi-step execution loops with state management.
• Ability to design multi-step agent execution loops with state management.

Responsibilities

• Build and operate multi-agent systems that solve user problems.
• Implement agentic coding workflows for autonomous iteration.
• Collaborate with Product & Design for shippable workflows.
• Collaborate with Product & Design to translate user problems into ship-ready agent workflows.
• Deploy and operate systems on cloud infrastructure using Docker and CI/CD pipelines.

Skills

Building LLM-powered systems

Backend engineering with Python or Node.js

Designing multi-step execution loops

Distributed systems primitives

Instrumenting and observing agent behavior

Deploying on cloud infrastructure

Inter-agent communication patterns

Implementing evaluation pipelines

Cost-aware execution strategies

RAG architectures

Tools

AWS

GCP

Docker

CI/CD pipelines

Job description

System1 is building the next generation of AI Agent–powered consumer products, and we’re looking for a Lead AI Product Engineer to help lead the effort. You’ll design, build, and operate our most ambitious AI agent systems from early prototypes through production, owning core architecture, reliability, and performance as these products scale.

This is a high‑impact, hands‑on engineering role for a builder who can move quickly from concept to working systems, shipping, iterating, and hardening real agent workflows that deliver consumer value at scale.

This role sits at the intersection of multi‑agent orchestration, agentic coding, and production engineering, offering the opportunity to work on cutting‑edge AI while directly influencing real‑world product outcomes and business performance.

You do not need prior experience with every protocol or framework listed below—we value strong systems thinking, production experience, and the ability to learn fast in emerging domains.

What You’ll Do:

• Build and operate multi‑agent systems that plan, coordinate, and execute tasks across multiple tools and services, to solve real user problems at scale.
• Implement agentic coding workflows, where agents generate code, run tests, create diffs or PRs, and iterate autonomously within guardrails.
• Design agent orchestration layers (planners, supervisors, evaluators) to manage task decomposition, retries, and long‑running execution.
• Instrument and monitor agent behavior across success, failure, latency, cost, and human overrides, with exposure to agent‑to‑tool payment flows as the ecosystem evolves.
• Collaborate with Product & Design to translate user problems into shippable agent workflows, iterating quickly based on real‑world feedback.

What You’ll Bring:

• Strong experience building LLM‑powered systems in production, including prompt engineering, structured outputs, tool/function calling, and agentic coding workflows where agents write, modify, and validate code within guardrails.
• Solid backend engineering skills with Python and/or Node.js, designing APIs, async services, and background workers that support long‑running, stateful agent workflows.
• Experience designing multi‑step, agent‑driven execution loops (plan → act → evaluate → retry), with explicit state management, idempotency, and failure recovery.
• Comfort with distributed systems primitives (queues, pub/sub, durable state, rate limiting, concurrency controls) required to coordinate agents at scale.
• Proven ability to instrument and observe agent behavior, emitting structured events for task success, failures, retries, overrides, and code‑generation outcomes.
• Experience deploying and operating systems on cloud infrastructure (AWS or GCP), using Docker and CI/CD pipelines.
• Familiarity with inter‑agent communication patterns, including message schemas, shared memory, blackboard systems, or event‑driven coordination.
• Experience implementing agent evaluation pipelines (prompt regression tests, simulation‑based evals, automated scoring, human‑in‑the‑loop review).
• Understanding of cost‑aware execution for agents (dynamic model routing, caching, batching, budget constraints).
• Experience with RAG architectures, vector databases, and persistent memory strategies.
• Exposure to emerging agent protocols (e.g., x402 payment flows, MCP), with the ability to implement evolving specs pragmatically.

Who You Are:

• You operate with high urgency, agency, and a bias for action, translating ideas into working systems quickly, learning from real‑world signals, and iterating continuously.
• You pair speed with rigor, bringing strong engineering fundamentals (systems thinking, evaluation discipline, reliability) so each iteration is meaningfully better, not just different.
• You embrace a “long is wrong” mindset, simplifying, automating, and removing bottlenecks to consistently reduce cycle time.
• You think and act like an owner, making sound tradeoffs and taking end‑to‑end accountability for performance, reliability, cost, and user experience.
• You optimize for technological leverage, building reusable primitives, tooling, and patterns that compound team output over time.
• You are growth‑minded, working backward from LTV/CAC and prioritizing engineering work that moves activation, retention, conversion, and cost per successful outcome.
• You excel at constraint removal, unblocking execution and increasing innovation per unit time, including through agentic tool use with clear guardrails and observability.

What We Have To Offer:

• Competitive salary + bonus + equity
• Generous PTO + 11 company holidays
• Open sick time
• 100% covered Medical, Dental, Vision for employees
• 401k with match
• Health & Dependent Care Flex Spending Account
• Paid professional development
• Leadership & growth opportunities
• Virtual company and team building events

System1’s headquarters is located in Marina del Rey, CA with an additional office in Bellevue, WA.

Employees near office locations are returning to the office.

Equal Employment Opportunity:

System1 provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.

This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation and training.