Quality Assurance Lead

Cloud Angles Digital Transformation; Hyderabad; On-site; INR 2,600,000 - 3,800,000; Full time

Job summary

Cloud Angles Digital Transformation in Hyderabad is seeking a QA Lead to own the QA strategy for AI-powered applications. You will lead testing across functional, API, performance, security, and AI-specific domains, mentoring a QA team and coordinating with product, engineering, and DevOps to elevate quality.

Responsibilities include building automated evaluation suites, defining QA processes, and delivering measurable quality metrics through dashboards.

Qualifications

• 8+ years of QA/testing experience.
• Experience creating enterprise-level test strategies.
• Strong experience with API, integration, functional, regression, and automation testing.
• Experience testing AI/ML/GenAI applications.
• Understanding of LLM evaluation and AI quality metrics.
• Experience with test automation frameworks.
• Strong SQL and API testing knowledge.
• Experience with tools such as Postman, Selenium, Playwright, Cypress, REST Assured, or equivalent.
• Strong understanding of CI/CD and automated testing pipelines.
• Strong defect management and analytical skills.

Responsibilities

• Define and own the overall test strategy and quality framework for AI applications.
• Establish test plans, testing methodologies, quality gates, and release criteria.
• Define QA processes across the complete software development lifecycle.
• Design test strategies for LLM, GenAI, RAG, and agentic AI applications.
• Define approaches for AI-specific testing including: Hallucination testing, Groundedness testing, Response relevance, Factuality, Consistency, Prompt robustness, Bias and safety testing, Guardrail testing, Agent/tool execution testing, Regression testing.
• Define AI evaluation datasets and test suites.
• Establish automated evaluation and regression frameworks.
• Review and approve test cases and test coverage.
• Define functional, integration, API, performance, security, and usability testing approaches.
• Establish defect management and root-cause analysis processes.
• Work closely with AI developers, product managers, UX, DevOps, and engineering teams.
• Define QA automation strategy and tooling.
• Establish quality metrics and dashboards.
• Lead QA reviews and mentor junior QA engineers.
• Drive continuous improvement of testing processes.

Skills

QA/testing experience

Software testing methodologies

Enterprise test strategy

API/integration/functional/regression/

AI/ML/GenAI testing

LLM evaluation/AI quality metrics

CI/CD and automated testing pipelines

Defect management & analytics

Mentoring QA teams

Test planning & requirements analysis

Tools

Postman

Selenium

Playwright

Cypress

REST Assured

Job description

We are looking for a QA Lead with 8+ years of software testing experience and strong exposure to testing modern AI-powered applications. The candidate will own the overall testing strategy for AI applications, establish quality frameworks, define testing methodologies, and lead QA activities across functional, integration, API, performance, security, and AI-specific testing.

Key Responsibilities

• Define and own the overall test strategy and quality framework for AI applications.
• Establish test plans, testing methodologies, quality gates, and release criteria.
• Define QA processes across the complete software development lifecycle.
• Design test strategies for LLM, GenAI, RAG, and agentic AI applications.
• Define approaches for AI-specific testing including: o Hallucination testing o Groundedness testing o Response relevance o Factuality o Consistency o Prompt robustness o Bias and safety testing o Guardrail testing o Agent/tool execution testing o Regression testing
• Define AI evaluation datasets and test suites.
• Establish automated evaluation and regression frameworks.
• Review and approve test cases and test coverage.
• Define functional, integration, API, performance, security, and usability testing approaches.
• Establish defect management and root-cause analysis processes.
• Work closely with AI developers, product managers, UX, DevOps, and engineering teams.
• Define QA automation strategy and tooling.
• Establish quality metrics and dashboards.
• Lead QA reviews and mentor junior QA engineers.
• Drive continuous improvement of testing processes.

Required Skills

• 8+ years of QA/testing experience.
• Strong understanding of software testing methodologies.
• Experience creating enterprise-level test strategies.
• Strong experience with API, integration, functional, regression, and automation testing.
• Experience testing AI/ML/GenAI applications.
• Understanding of LLM evaluation and AI quality metrics.
• Experience with test automation frameworks.
• Strong SQL and API testing knowledge.
• Experience with tools such as Postman, Selenium, Playwright, Cypress, REST Assured, or equivalent.
• Strong understanding of CI/CD and automated testing pipelines.
• Strong defect management and analytical skills.

Preferred Skills

• Experience with RAG and LLM application testing.
• Experience with tools such as Ragas, DeepEval, Promptfoo, LangSmith, or equivalent.
• Python scripting experience.
• Experience testing AI agents and tool-calling workflows.
• Experience with performance and load testing.
• Experience with cloud-based applications.
• Experience leading QA teams.