Software Engineer I - LLM Observability

Ladders; United States; Remote; USD 160,000 - 180,000; Full time

Benefits offered by this job

Generous equity

Unlimited PTO

Remote-friendly culture

Home office stipend

Full benefits for dependents

Parental leave

Annual development allowance

Job summary

Ladders is seeking a Software Engineer I - LLM Observability to help build observability capabilities for AI-centered systems, delivering insights on cost, errors, and signals across conversations.

You will work across the stack from data ingestion to frontend visuals, collaborating with product and design to create intuitive tools that help customers monitor AI behavior and performance.

Qualifications

• 3+ years of software engineering experience.
• Experience shipping customer-facing features at a SaaS company.
• Full-stack proficiency in Go or Python and frontend in TypeScript/React.
• Track record of collaboration with product managers and designers.
• Hands-on use of AI tools in engineering work.
• Experience being on call, with rotation after 90 days.
• Bonus: Experience with LLM or agent-powered features.
• Bonus: Familiarity with observability tooling like OpenTelemetry.

Responsibilities

• Deliver AI observability capabilities from ingestion to visualizations.
• Contribute to the data model and infrastructure for agent observation.
• Build systems to analyze AI-specific signals such as cost and errors.
• Collaborate with product and design to ensure usability.
• Share insights on AI system behavior across conversations.
• Iterate based on feedback loops and share outcomes.

Skills

3+ years experience

Backend Go/Python

Frontend TS/React

Collaboration with product/design

AI tooling in engineering

On-call rotation

Ambiguity navigation

LLM/agent features

Tools

OpenTelemetry

Job description

For our client, we are seeking a Software Engineer I - LLM Observability to join the team of a leader in the Technical Services space.

This role will help build and evolve observability capabilities for AI and LLM-driven systems, with a focus on making complex behavior visible, understandable, and actionable. You will contribute across the stack, from data ingestion and system infrastructure through user-facing experiences and visualizations. The work will involve close collaboration with product and design partners to create intuitive tools that help customers monitor cost, errors, and other critical signals across AI workflows.

Location: Remote - US based candidates only, no visa sponsorship available

Compensation: $160,000 – $180,000 annually

Responsibilities

• Deliver AI observability capabilities from ingestion to visualizations
• Contribute to the data model and infrastructure for agent observation
• Build systems to analyze AI-specific signals such as cost and errors
• Collaborate with product and design to ensure usability
• Share insights on AI system behavior across conversations
• Iterate based on feedback loops and share outcomes

Qualifications

• 3+ years of software engineering experience
• Experience shipping customer-facing features at a SaaS company
• Full-stack proficiency in backend development using Go or Python and frontend development using TypeScript and React
• Track record of collaboration with product managers and designers
• Hands-on use of AI tools in engineering work
• Experience being on call, with rotation participation after 90 days
• Ability to navigate ambiguity in a fast-paced environment
• Bonus: Experience with LLM or agent-powered features
• Bonus: Familiarity with observability tooling like OpenTelemetry

Benefits

• Generous equity with an employee-friendly stock program
• Transparent pay structure based on experience
• Unlimited PTO for time to recharge
• Distributed-first culture
• Home office, co-working, and internet stipend
• Full benefits coverage for employees and dependents
• Up to 16 weeks of paid parental leave
• Annual development allowance
• And much more...

Our client is an equal opportunity employer. We encourage you to apply even if you don’t meet every qualification—your background could be exactly what this team needs.