Senior AI Agent Engineer — Observability Platform

Logoslabs; San Francisco (CA); Remote; USD 183,000 - 206,000; Full time

Benefits offered by this job

Equity stake

Unlimited PTO

Home office stipend

Full benefits

Parental leave

Development allowance

Job summary

Honeycomb is building AI agents that investigate, reason, and act on real observability data inside Canvas. You will own the agent work end-to-end, shipping production-grade capabilities and evaluating improvements across Canvas, the MCP server, and Canvas Skills.

We value deep agent expertise, judgment on design shifts, and the ability to prototype next steps while keeping systems reliable and observable. This is a fully distributed, remote role with strong equity and benefits.

Qualifications

• AI and agent engineering experience in production systems.
• Demonstrated ownership of end-to-end features from design to production.
• Informed opinions on agent design shifts in recent months.
• Depth in agent architecture and data-store capabilities.
• Ability to forecast product directions with prototypes.

Responsibilities

• Design and deliver production-grade agents that investigate, reason, and act on live observability data inside Canvas.
• Own the agent work; scope, build, ship, and maintain the agents including evals.
• Build agents that leverage high-cardinality data stores to reason over signal.
• Extend Canvas, MCP server, and Canvas Skills with new capabilities and improvements.
• Define meaningful success metrics and ensure maintainability.

Skills

AI engineering

End-to-end ownership

Current judgment

Agent architecture

Product judgment

Tools

Agent tooling

RAG

Prompt engineering

Job description

Honeycomb is building AI agents that investigate, reason, and act on real observability data inside Canvas. You will own the agent work end-to-end, shipping production-grade capabilities and evaluating improvements across Canvas, the MCP server, and Canvas Skills.

We value deep agent expertise, judgment on design shifts, and the ability to prototype next steps while keeping systems reliable and observable. This is a fully distributed, remote role with strong equity and benefits.