LLM Observability Engineer - Remote AI Signals & Debugging

Honeycomb.io; Canada; Remote; CAD 219,000 - 246,000; Full time

Benefits offered by this job

Equity program

Unlimited PTO

Home office stipend

Full benefits

Parental leave (up to 16 weeks)

Development allowance

Distributed-first culture

Job summary

Honeycomb.io is expanding its AI observability team, building end-to-end ingestion and in-product visualizations for AI agents. You will shape data models, surface AI signals, and collaborate with product and design to deliver usable features.

The role requires 3+ years of software engineering, backend Go/Python and frontend TypeScript/React, and hands-on AI tool experience. Remote-friendly, with competitive base salary and strong equity.

Qualifications

• 3+ years software engineering experience.
• Experience shipping customer-facing features at a SaaS company.
• Full-stack proficiency with Go and/or Python backend and TypeScript/React frontend.
• Regular, hands-on use of AI tools in daily engineering work.
• Experience being on call (rotation after ~90 days).
• Ability to navigate ambiguity in a fast-moving field.

Responsibilities

• Deliver AI observability capabilities end to end, from ingestion to in-product visualizations.
• Contribute to data model and infrastructure to observe agents across the product.
• Build systems that surface AI-specific signals such as cost, token usage, errors, and anomalies.
• Collaborate with product and design to ensure usable, intuitive features.
• Share learnings about AI system behavior across conversations and workloads.
• Iterate with fast feedback loops and experimentation, sharing wins and failures.

Skills

Go / Python backend

TypeScript / React frontend

Full-stack engineering

AI tools usage

On-call rotation

Tools

OpenTelemetry

Observability tooling

Job description

Honeycomb.io is expanding its AI observability team, building end-to-end ingestion and in-product visualizations for AI agents. You will shape data models, surface AI signals, and collaborate with product and design to deliver usable features.

The role requires 3+ years of software engineering, backend Go/Python and frontend TypeScript/React, and hands-on AI tool experience. Remote-friendly, with competitive base salary and strong equity.