Software Engineer, Conversational Products & Agent Workflows

The Briefed; Boston (MA); On-site; USD 140,000 - 200,000; Full time

Job summary

Klaviyo is seeking a Software Engineer, Conversational Products & Agent Workflows, to join the Customer Agent team. You will build scalable backend services and AI-driven experiences that enable autonomous agents to retrieve context, execute tools, and perform multi-step actions for customers.

In this backend-heavy role, you’ll work on RAG, grounding, evaluation, and reliability, while participating in on-call rotations and collaborating with customers to translate feedback into product

Qualifications

• 3+ years of backend/distributed application experience.
• Hands-on with generative AI or agentic AI in production or user-facing contexts.
• Strong Python and modern backend framework experience.

Responsibilities

• Build and improve reliable backend services and APIs powering AI-driven customer experiences.
• Develop agentic features including tool orchestration and multi-step agent execution.
• Contribute to RAG, grounding, and model selection for AI workflows.
• Create evaluation suites and production-quality signals for AI features.
• Implement production guardrails, observability, and failure handling for agentic features.
• Engineer asynchronous and distributed task processing for high-volume workloads.
• Engage with customers to translate operational feedback into feature improvements.

Skills

Backend development

Distributed systems

Async processing

Evaluation & QA

Python programming

Tools

Python

FastAPI

Django

Celery

Kafka

AWS SQS

RabbitMQ

Redis

Docker

Kubernetes

CI/CD

SQL

NoSQL

Job description

At Klaviyo, we believe the future of software lies not only in tools that help people work efficiently, but in intelligent systems that can take action, learn from outcomes, and improve customer experiences over time. Serving over 167,000 customers and processing billions of consumer profiles and conversion signals, Klaviyo provides a unique environment for building state-of-the-art AI systems at scale.

We are looking for a Software Engineer, Conversational Products & Agent Workflows to join the Customer Agent team—Klaviyo’s AI-native conversational platform. In this backend-heavy role, you will build scalable services and product experiences that enable autonomous agents to retrieve context, execute tools, and carry out reliable multi-step actions on behalf of customers.

Key Responsibilities

• Backend & API Engineering: Build and improve reliable backend services, data pipelines, and REST/gRPC APIs that power AI-driven customer experiences.
• Agentic Features & Workflows: Develop production agentic capabilities involving tool orchestration, context retrieval, structured outputs, and multi-step agent execution.
• Applied AI & RAG: Contribute to solving applied AI challenges across Retrieval-Augmented Generation (RAG), grounding, hallucination mitigation, model selection, and hybrid/deterministic logic vs. LLM execution.
• Evaluation & Quality: Build and apply evaluation suites using representative datasets, automated/human feedback, regression testing, and production quality signals.
• System Reliability: Implement production guardrails, fallbacks, retry policies, observability metrics, and failure-handling mechanisms for agentic features.
• Distributed Workflows: Engineer asynchronous processing workflows and distributed task queues capable of handling high-volume AI workloads.
• Operations & Customer Feedback: Participate in the team on-call rotation to mitigate production incidents, and engage with external customers to translate operational feedback into feature improvements.

Experience & Education

• 3+ years of professional software engineering experience, with a focus on backend development or distributed applications.
• Hands-on experience building or contributing to generative AI or agentic AI applications (ideally used by real users or in production environments).

Technical Capabilities

• Proficiency in Python and experience with modern backend frameworks (e.g., FastAPI, Django).
• Practical experience with asynchronous task processing and event-driven systems (Celery, Kafka, AWS SQS, RabbitMQ, or Redis).
• Working knowledge of database modeling, SQL/NoSQL persistence patterns, and API architecture.
• Comfort operating in cloud-native environments (AWS, Docker/containers, Kubernetes, infrastructure automation, CI/CD pipelines).
• Ability to evaluate technical tradeoffs across quality, latency, cost, reliability, and architectural complexity.

Preferred Qualifications

• Experience training, fine-tuning, distilling, or adapting machine learning and language models for production workflows.
• Hands-on experience with reinforcement learning or feedback-driven optimization techniques.
• Exposure to evaluation infrastructure (e.g., LLM-as-judge frameworks, evaluator calibration, benchmark datasets, AI quality tooling).
• Experience operating AI systems at scale or optimizing inference cost, latency, or throughput.