Senior AI Observability Engineer — SaaS, Full‑Stack

honeycomb; United States; Remote; USD 160,000 - 180,000; Full time

Benefits offered by this job

Equity

Unlimited PTO

Health coverage

Parental leave

Development allowance

Job summary

Honeycomb is seeking an engineer to build observability features that help users understand AI agents, their costs, and where they fail. Work spans from data models to customer-facing tools for exploring agent conversations and workloads.

You'll join a small cross-functional team and ship capabilities used in production, with on-call duties starting after ~90 days. Collaboration with product and design is essential to shape usable workflows for customers.

Qualifications

• 3+ years of software engineering experience, including shipping customer-facing SaaS features.
• Full-stack development with backend in Go and/or Python and frontend in TypeScript/React.
• Experience collaborating with product managers and designers.
• Regular hands-on use of AI tools in engineering work.
• Comfort with on-call support and changing requirements.

Responsibilities

• Develop AI observability features end to end, from data ingestion to product visualizations.
• Improve the data model and infrastructure for observing agents and their surrounding systems.
• Analyze and surface signals such as token usage, costs, errors, and unusual behavior.
• Partner with product and design to shape usable workflows around customer needs.
• Share findings about how users investigate AI behavior, and iterate through experiments and feedback.
• Participate in an on-call rotation, expected to begin after roughly 90 days.

Skills

Go

Python

TypeScript

React

On-call experience

Collaborate with product/design

Tools

OpenTelemetry

Job description

Honeycomb is seeking an engineer to build observability features that help users understand AI agents, their costs, and where they fail. Work spans from data models to customer-facing tools for exploring agent conversations and workloads.

You'll join a small cross-functional team and ship capabilities used in production, with on-call duties starting after ~90 days. Collaboration with product and design is essential to shape usable workflows for customers.