Head of AI Agent Evaluation & Instrumentation

Singtel Group; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Singtel is hiring a Lead for Agent Evaluation & Instrumentation to become the independent quality authority for the telco’s AI estate. You will own the Day 1–Day 2 handover, define metrics and instrumentation, and ensure safe, reliable production AI deployments.

You will lead a team of engineers and analysts, maintain independence, chair gate reviews, and drive improvements through telemetry and dashboards in collaboration with multiple internal partners.

Qualifications

• Degree in Computer Science, Data Science, Engineering, or related field
• 10+ years in software/ML/quality with 3+ years leading technical teams
• Experience operating or evaluating ML/LLM systems in production
• Metrics design and observability/instrumentation for AI systems
• Understanding of RAG, SQL agents, autonomous agents and AI/ML model lifecycles
• Strong stakeholder management and the independence to hold a line
• Clear written communication for gate findings and executive reporting
• People leadership and coaching

Responsibilities

• Lead the Agent Evaluation & Instrumentation function as the independent quality authority for the telco’s production AI estate.
• Own the Day 1 → Day 2 handover process and gate authority over every AI agent and AI/ML model entering production.
• Define metrics catalogue, evaluation suites, and instrumentation standards, proving agent quality, safety, and reliability.
• Lead a team of evaluation and instrumentation engineers and analysts, safeguarding independence from delivery pressure.
• Chair operability gate reviews and issue independent go/no-go decisions with findings; track remediation to closure.
• Set instrumentation and telemetry standards and certify telemetry completeness before shadow-run exit.
• Lead, coach the team; build AI/Agent dashboards in consultation with other services and product owners.
• Drive the improvement engine by routing production signals into a prioritised backlog and evidencing improvements.
• Represent the function with cross-functional partners and provide quality inputs to reporting.

Education

Degree in Computer Science, Data Science, Engineering, or related field

Job description

Singtel is hiring a Lead for Agent Evaluation & Instrumentation to become the independent quality authority for the telco’s AI estate. You will own the Day 1–Day 2 handover, define metrics and instrumentation, and ensure safe, reliable production AI deployments.

You will lead a team of engineers and analysts, maintain independence, chair gate reviews, and drive improvements through telemetry and dashboards in collaboration with multiple internal partners.