Senior Customer Experience Operations manager

Saviynt; Bengaluru; On-site; INR 1,500,000 - 2,300,000; Full time

Job summary

Saviynt is seeking an AI Performance Analyst in Bengaluru to optimize performance analytics for their AI agent fleet. The role focuses on tracking and enhancing deflection rates, closure rates, and agent confidence metrics, providing insights to improve support operations.

Candidates should have 3–6 years of experience in analytics, especially in AI contexts, and demonstrate strong analytical skills along with effective communication abilities to convey complex data insights.

Qualifications

• 3–6 years of experience in analytics or business intelligence, preferably with AI/ML systems.
• Experience in designing rubrics and running QA cycles.
• Ability to analyze AI performance and measure real-world model behavior.
• Clear communication to turn AI performance analysis into actionable recommendations.
• Familiarity with prompt or context engineering to assess agent responses.

Responsibilities

• Track AI agent performance metrics and productivity gains.
• Identify issues causing escalations and suggest improvements.
• Build real-time dashboards for performance monitoring.
• Build and maintain real-time dashboards showing AI health, case closure velocity, and deflection trends for leadership.
• Run structured QA and eval cycles on AI outputs — assess accuracy, tone, and resolution completeness to prevent quality drift.
• Prepare concise briefings for leadership on AI performance and productivity gains with clear investment recommendations.

Skills

Analytics

AI/ML systems analysis

Communication

Quality Assurance

Prompt engineering

Job description

Saviynt's AI-powered identity platform manages and governs human and non-human access to all of an organization's applications, data, and business processes. Customers trust Saviynt to safeguard their digital assets, drive operational efficiency, and reduce compliance costs. Built for the AI age, Saviynt is today helping organizations safely accelerate their deployment and usage of AI. Saviynt is recognized as the leader in identity security, with solutions that protect and empower the world’s leading brands, Fortune 500 companies and government institutions. For more information, please visit www.saviynt.com.

About the Role

At Saviynt, our customer support runs on autonomous AI agents that triage, route, and resolve cases without human intervention. This role sits at the center of that mission. You are the analytical engine that tells us exactly where our agents are winning, where they’re failing, and what it will take to push autonomous resolution rates higher. You track agent performance at a granular level, identify patterns in cases that escape to humans, and feed those findings directly into the tuning cycles that make our AI sharper over time. We need a specialist who goes beyond surface-level reporting to rigorously interrogate deflection and closure metrics. As case patterns evolve, you will benchmark agent confidence scores and identify the precise improvements needed to optimize our autonomous capabilities. The insights you surface will directly shape how our agents are trained, how AI-assisted productivity gains are measured, and how we define what “good” looks like in an AI-first support operation.

What you will be doing

• Own the performance analytics for our AI agent fleet — tracking deflection rates, autonomous closure rates, escalation triggers, confidence score distributions, and CSAT by resolution type to give us a precise, real-time picture of where agents are succeeding and where they’re not.
• Dig into cases that escape AI resolution — identifying the intent clusters, knowledge gaps, and routing failures that cause unnecessary escalations, and translating those findings into specific improvements for agent training and prompt refinement.
• Define and govern the KPI framework for AI productivity — including human-time-saved per resolved case, agent-assisted vs. fully autonomous closure ratios, and the quality metrics that validate whether AI resolution is genuinely satisfying customers or just closing tickets.
• Build and maintain real-time dashboards that surface AI agent health, case closure velocity, and deflection trends — giving the support leadership a live signal on how the AI operation is performing at any given moment.
• Run structured QA and eval cycles on AI agent outputs — reviewing response accuracy, tone calibration, and resolution completeness to catch quality drift before it shows up in CSAT or re-open rates.
• Prepare concise, evidence-based briefings for leadership on AI agent performance and productivity gains — translating what the data shows into clear narratives about where to invest next to push resolution rates higher.

What you bring

• 3–6 years of experience in analytics or business intelligence, with at least some of that time spent analyzing AI, ML, or LLM powered systems in a production environment where you have measured how models behave in the real world.
• Hands on experience with AI agent evaluation; you know how to design rubrics, run QA cycles, and define what “good resolution” actually means in a support context. You can calibrate quality, not just count tickets.
• Sharp analytical instincts; you can tell the difference between a deflection rate that’s improving because agents are getting better and one that’s improving because frustrated users are giving up. You ask the next question, not just the first one.
• Clear communication; you can turn a complex AI performance analysis into a crisp recommendation that a support leader can act on immediately, without needing a technical deep‑dive to understand the point.
• Familiarity with prompt engineering or context engineering enough to have an informed opinion on why an agent responded the way it did and what a better prompt structure would look like.

₹0 - ₹0 a year

Compliance and Security Requirements

• Complete security & privacy literacy and awareness training during onboarding and annually thereafter.
• Review and adhere to information security/privacy policies and procedures, such as Data Classification, Retention & Handling Policy; Incident Response Policy/Procedures; Business Continuity/Disaster Recovery Policy/Procedures; Mobile Device Policy; Account Management Policy; Access Control Policy; Personnel Security Policy; Privacy Policy.

Saviynt is an equal opportunity employer and we welcome everyone to our team. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status.