Software Engineer

Tata Consultancy Services; Hyderabad; On-site; INR 1,500,000 - 2,300,000; Full time

Job summary

Tata Consultancy Services is seeking an experienced AI Test Engineer to drive QA for AI-powered applications, including LLM models, conversational bots, autonomous agents, and MCP-enabled systems. Key focus areas include Playwright with .NET, API/UI automation, AI/LLM validation, and CI/CD automation for scalable test frameworks.

Location is PAN India with 5–12 years of experience. You will design strategies for LLM-powered apps, validate AI responses, test agent workflows, and ensure

Qualifications

• Experience in validating AI/LLM powered applications and agents.
• Strong API/UI automation testing skills across CI/CD pipelines.
• Hands-on with Playwright and .NET for test automation.
• Familiarity with AI validation frameworks and test data generation.

Responsibilities

• Design and execute test strategies for LLM-powered apps, AI chatbots, conversational AI, and autonomous agents.
• Develop and maintain Playwright (.NET) automation frameworks for UI and API testing.
• Test MCP endpoints and MCP-enabled agent workflows.
• Automate API testing and integrate test suites into CI/CD pipelines.
• Create test datasets, evaluation metrics, and AI validation frameworks.
• Perform regression, functional, integration, and end-to-end testing for AI systems.
• Collaborate with developers, data scientists, and product teams to ensure quality delivery.
• Monitor and troubleshoot AI system failures, agent interactions, and model behavior.
• Implement quality gates within Azure DevOps/GitHub Actions/Jenkins pipelines.

Skills

Playwright with .NET

API testing

UI automation

CI/CD automation

Tools

Playwright (.NET)

Azure DevOps

GitHub Actions

Jenkins

Job description

We are looking an experienced AI Test Engineer to drive quality assurance for AI-powered applications, including LLM models, conversational bots, autonomous agents, and MCP-enabled systems. The ideal candidate will have strong expertise in Playwright with .NET, API/UI automation, AI/LLM validation, and CI/CD automation. The role involves building scalable test frameworks, validating AI responses, testing agent workflows, and ensuring the reliability, security, and performance of GenAI solutions.

Location: PAN India

Experience Range: 5 to 12years

Key Responsibilities:

• Design and execute test strategies for LLM-powered applications, AI chatbots, conversational AI, and autonomous agents.
• Develop and maintain Playwright (.NET) automation frameworks for UI and API testing.
• Accuracy
• Relevance
• Hallucinations
• Consistency
• Safety and Guardrails
• Test MCP (Model Context Protocol) endpoints and MCP-enabled agent workflows.
• Automate API testing and integrate test suites into CI/CD pipelines.
• Create test datasets, evaluation metrics, and AI validation frameworks.
• Perform regression, functional, integration, and end-to-end testing for AI systems.
• Collaborate with developers, data scientists, and product teams to ensure quality delivery.
• Monitor and troubleshoot AI system failures, agent interactions, and model behavior.
• Implement quality gates within Azure DevOps/GitHub Actions/Jenkins pipelines.