Head of AI Agent Quality & Instrumentation

Singtel; Singapore; On-site; Confidential; Full time

Job summary

Singtel is building an AI-first telco and seeks a senior leader to own Agent Evaluation & Instrumentation. You will govern quality for production AI estates, define success metrics, and ensure go/no-go decisions for AI agents and models.

You will lead a team of engineers and analysts, champion independent evaluation, and collaborate with global partners to raise reliability and safety. In this role you will shape telemetry, dashboards, and operational dashboards across agent archetypes, drive

Qualifications

• Degree in Computer Science, Data Science, Engineering, or related field.
• 10+ years in software/ML/quality with 3+ years leading technical teams.
• Experience operating or evaluating ML/LLM systems in production.
• Metrics design and observability/instrumentation for AI systems.
• Understanding of RAG, SQL agents, autonomous agents and AI/ML model lifecycles.
• Strong stakeholder management and the independence to hold a line.
• Clear written communication for gate findings and executive reporting.

Responsibilities

• Lead the Agent Evaluation & Instrumentation function as the independent quality authority for production AI estate.
• Own the Day 1 → Day 2 handover process and go/no-go gate authority over every AI agent and AI/ML model entering production.
• Define what “good” means through metrics catalog, evaluation suites, and instrumentation standards.
• Lead a team of evaluation and instrumentation engineers and analysts, safeguarding independence from delivery pressure.
• Set instrumentation and telemetry standards and certify telemetry completeness before shadow-run exit.
• Lead, coach the team to build AI/Agent dashboards in consultation with stakeholders.
• Drive the improvement engine by translating production signals into a prioritised backlog.
• Represent with cross-functional partners and provide quality inputs to reporting.
• Lead and develop the team; manage independence, workload prioritisation, and performance.

Skills

Experience in ML/LLM systems

Metrics design

Observability/instrumentation

Stakeholder management

Gate findings & executive reporting

People leadership & coaching

Strong written communication

Education

Degree in Computer Science / Data Science / Engineering

Job description

Singtel is building an AI-first telco and seeks a senior leader to own Agent Evaluation & Instrumentation. You will govern quality for production AI estates, define success metrics, and ensure go/no-go decisions for AI agents and models.

You will lead a team of engineers and analysts, champion independent evaluation, and collaborate with global partners to raise reliability and safety. In this role you will shape telemetry, dashboards, and operational dashboards across agent archetypes, drive