GenAI Observability Program Lead
Uber
An application made for this job — a tailored resume and cover letter that speak straight to the posting.
Benefits offered by this job
Job summary
Uber is seeking an AI Observability Program Leader to own the end-to-end strategy, design, and implementation of frameworks that monitor, understand, and improve Uber's GenAI-powered agentic systems. You will bridge raw AI logs to actionable product insights and define micrometrics that pinpoint where reasoning succeeds or fails.
The role partners with Product, Engineering, and Data Science to translate complex agent behaviors into actionable signals, driving reliability and CX improvements
Qualifications
- 5+ years of experience in Technical Program Management, Product Operations, AI Quality, or Observability.
- Bachelor's degree in Engineering, Computer Science, Data Science, or a related technical field.
Responsibilities
- Architect Observability Frameworks to understand AI agentic reasoning and enable deep analysis of step-by-step decisions.
- Drive Autoeval Strategy with automated evaluation systems for scalable AI performance insights.
- Define Micrometrics to identify latent failures and improve products.
- Lead Pre-Launch Simulation with Product & Engineering to test agents against edge cases before deployment.
- Coordinate Cross-Functional Technical Partnership to embed observability into the development lifecycle.
Skills
Education
Tools
Job description
Uber is seeking an AI Observability Program Leader to own the end-to-end strategy, design, and implementation of frameworks that monitor, understand, and improve Uber's GenAI-powered agentic systems. You will bridge raw AI logs to actionable product insights and define micrometrics that pinpoint where reasoning succeeds or fails.
The role partners with Product, Engineering, and Data Science to translate complex agent behaviors into actionable signals, driving reliability and CX improvements
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
