Remote Software Engineer I, LLM Observability & AI Metrics
Ladders
Get a reply from this employer
Your resume and cover letter, tailored to exactly what they’re hiring for.
Benefits offered by this job
Job summary
Ladders is seeking a Software Engineer I - LLM Observability to help build observability capabilities for AI-centered systems, delivering insights on cost, errors, and signals across conversations.
You will work across the stack from data ingestion to frontend visuals, collaborating with product and design to create intuitive tools that help customers monitor AI behavior and performance.
Qualifications
- 3+ years of software engineering experience.
- Experience shipping customer-facing features at a SaaS company.
- Full-stack proficiency in Go or Python and frontend in TypeScript/React.
- Track record of collaboration with product managers and designers.
- Hands-on use of AI tools in engineering work.
- Experience being on call, with rotation after 90 days.
- Bonus: Experience with LLM or agent-powered features.
- Bonus: Familiarity with observability tooling like OpenTelemetry.
Responsibilities
- Deliver AI observability capabilities from ingestion to visualizations.
- Contribute to the data model and infrastructure for agent observation.
- Build systems to analyze AI-specific signals such as cost and errors.
- Collaborate with product and design to ensure usability.
- Share insights on AI system behavior across conversations.
- Iterate based on feedback loops and share outcomes.
Skills
Tools
Job description
Ladders is seeking a Software Engineer I - LLM Observability to help build observability capabilities for AI-centered systems, delivering insights on cost, errors, and signals across conversations.
You will work across the stack from data ingestion to frontend visuals, collaborating with product and design to create intuitive tools that help customers monitor AI behavior and performance.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
