AI Observability Engineer

Tata Consultancy Services; Hyderabad, Bengaluru; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Tata Consultancy Services is seeking an experienced AI Model Monitoring & Observability expert to design and implement observability for AI/ML and GenAI apps. You will define KPIs, monitor drift, and ensure reliable AI services.

Join a function collaborating with data scientists and ML engineers to deploy monitoring, governance, and incident RCA processes, driving Responsible AI practices in production systems.

Qualifications

• Bachelor's or Master's in Computer Science, Data Science, Engineering.
• 5+ years in Data Engineering, AI/ML, MLOps, or Platform Engineering.
• 2+ years working with GenAI/LLM solutions.
• Experience supporting production AI applications.

Responsibilities

• Design observability frameworks for AI/ML and GenAI applications.
• Monitor model performance, prediction quality, latency, throughput, and availability.
• Track model drift, data drift, concept drift, and performance degradation.
• Establish AI-centric KPIs, SLAs, and SLOs.
• Collaborate with data scientists and ML engineers to deploy monitoring solutions.
• Ensure compliance with Responsible AI and governance policies.
• Investigate AI service outages and model failures; perform RCA.

Skills

Data Engineering

AI/ML

MLOps

Platform Engineering

Education

Bachelor's or Master's in Computer Science, Data Science, Engineering

Job description

Role & responsibilities

AI Model Monitoring & Observability

• Design and implement observability frameworks for AI/ML and GenAI applications.
• Monitor model performance, prediction quality, latency, throughput, and availability.
• Track model drift, data drift, concept drift, and performance degradation.
• Establish AI-centric KPIs, SLAs, and SLOs.

Generative AI Observability

• Monitor LLM applications for:
• Hallucinations
• Response quality
• Prompt effectiveness
• Toxicity detection
• Cost and token consumption
• Retrieval quality in RAG systems

MLOps & Governance

• Collaborate with data scientists and ML engineers to deploy monitoring solutions.
• Ensure compliance with Responsible AI and governance policies.

Incident Management

• Investigate AI service outages and model failures.
• Perform root cause analysis (RCA).

Preferred candidate profile

• Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or related field.
• 5+ years in Data Engineering, AI/ML, MLOps, or Platform Engineering.
• 2+ years working with GenAI/LLM solutions.
• Experience supporting production AI applications.