AI Governance & Operations Specialist

Authentic Staffing Agency; Mumbai; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Authentic Staffing Agency seeks an AI Chatbot Governance & Operations Specialist to manage enterprise conversational AI platforms, ensuring performance, compliance, reliability, and continuous improvement.

Role focuses on chatbot monitoring, governance, risk management, analytics, cost optimization, and stakeholder coordination.

Qualifications

• 4+ years in Conversational AI Operations or AI Governance.
• Hands-on experience with OpenAI GPT, Microsoft Copilot, Google Gemini, AWS Bedrock, or Salesforce Einstein Bots.
• Experience supporting enterprise customer service or employee support chatbots.
• Hands-on experience with LLMs, AI orchestration frameworks, RAG, vector databases and AI agents.
• Experience in cloud platforms (AWS/Azure), microservices and API architecture.
• Experience delivering customer service, sales, service operations or conversational AI platforms.

Responsibilities

• Monitor chatbot performance, availability, response quality, and user experience across channels.
• Track key metrics including Resolution Rate, Escalation Rate, CSAT, Response Accuracy, and Hallucination Rate.
• Implement and monitor AI guardrails to ensure safe, compliant, and responsible AI behaviour.
• Identify and mitigate risks related to hallucinations, prompt injection, data privacy, security, and policy violations.
• Analyse chatbot conversations and feedback to identify improvement opportunities.
• Collaborate with development teams to enhance chatbot effectiveness and user experience.
• Monitor LLM usage, token consumption, and AI operational costs, recommending optimization strategies.
• Develop dashboards and reports on chatbot performance, governance, and utilization metrics.
• Coordinate with Product, IT, Security, Legal, and Compliance teams to support governance reviews and release management.

Tools

SQL

API Integrations

AWS

Azure

MLOps

Observability

Job description

Role & responsibilities

Role Summary

We are seeking an AI Chatbot Governance & Operations Specialist to manage enterprise conversational AI platforms and ensure their performance, compliance, reliability, and continuous improvement. The role focuses on chatbot monitoring, AI governance, risk management, analytics, cost optimization, and stakeholder coordination.

Key Responsibilities

AI Chatbot Operations & Performance Monitoring

• Monitor chatbot performance, availability, response quality, and user experience across channels.
• Track key metrics including Resolution Rate, Escalation Rate, CSAT,

Response Accuracy, and Hallucination Rate.

AI Governance & Risk Management

• Implement and monitor AI guardrails to ensure safe, compliant, and responsible AI behaviour.
• Identify and mitigate risks related to hallucinations, prompt injection, data privacy, security, and policy violations.

Continuous Improvement & Feedback Management

• Analyse chatbot conversations and feedback to identify improvement opportunities.
• Collaborate with development teams to enhance chatbot effectiveness and user experience.

Token Consumption & Cost Management

• Monitor LLM usage, token consumption, and AI operational costs, recommending optimization strategies.

Reporting & Stakeholder Management

• Develop dashboards and reports on chatbot performance, governance, and utilization metrics.

General

• Coordinate with Product, IT, Security, Legal, and Compliance teams to support governance reviews and release management.

Technical Skills

Conversational AI Platforms & Generative AI (LLMs)

• LLMs (OpenAI, Azure OpenAI, Gemini, Claude, Bedrock)

Prompt Engineering & Knowledge Management

AI Governance, Responsible AI & Compliance

AI Risk Management, Hallucination Monitoring & Security

AI Monitoring, Analytics & Performance Reporting

SQL, API Integrations, AWS and Architecture

RAG, Knowledge Graphs & Vector Databases

Python, APIs, Microservices

Salesforce Ecosystem

CI/CD, MLOps, Observability

Preferred Experience

4+ years in Conversational AI Operations, AI Governance, AI Product Support, or

related roles.

Hands-on experience with OpenAI GPT, Microsoft Copilot, Google Gemini, AWS Bedrock, or Salesforce Einstein Bots.

Experience supporting enterprise customer service or employee support chatbots.

Hands-on experience with LLMs, AI orchestration frameworks, RAG, vector databases and AI agents.

Experience in cloud platforms (AWS/Azure), microservices and API architecture.

Experience delivering customer service, sales, service operations or conversational AI platforms.

Competencies

Analytical Thinking

AI Governance Mindset

Problem Solving

Stakeholder Management

Communication & Presentation Skills

Continuous Improvement Orientation