Senior AI Agent Observability Engineer

Honeycomb; United States; On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

100% covered medical, dental, vision

Unlimited PTO

Sabbatical program

401K retirement plans

Paid parental leave

Family-planning support

Annual development stipend

Home setup stipend

Company equity

Job summary

Honeycomb is seeking a software engineer to advance AI observability capabilities across ingestion, storage, and in-product visualizations. You will work in a small, cross-functional team with PMs and designers to deliver production features that help customers debug and optimize AI agents and workloads.

The role requires 3+ years of software engineering, experience shipping customer-facing SaaS features, and strong full-stack skills in Go/Python and TypeScript/React.

Qualifications

• 3+ years of software engineering experience.
• Experience shipping customer-facing features at a SaaS company.
• Experience being on call with a rotation after ~90 days.
• Full-stack development with backend (Go/Python) and frontend (TypeScript/React).
• Bonus: hands-on observability tooling such as OpenTelemetry.

Responsibilities

• Deliver AI observability capabilities end to end, from ingestion to in-product visualizations.
• Contribute to data model and infrastructure to observe agents and their ecosystems.
• Build systems that analyze AI signals: cost, token usage, errors, anomalies.
• Collaborate with product and design to ensure usable, intuitive features.
• Iterate with fast feedback loops and share learnings from experiments.

Skills

Go

Python

TypeScript

React

On-call experience

SaaS experience

AI observability

Tools

OpenTelemetry

Job description

Honeycomb is seeking a software engineer to advance AI observability capabilities across ingestion, storage, and in-product visualizations. You will work in a small, cross-functional team with PMs and designers to deliver production features that help customers debug and optimize AI agents and workloads.

The role requires 3+ years of software engineering, experience shipping customer-facing SaaS features, and strong full-stack skills in Go/Python and TypeScript/React.