Python Automation Engineer

Mployee.me; Mumbai; On-site; INR 3,000,000 - 4,200,000; Full time

Job summary

Deloitte’s Technology & Transformation practice is seeking a Senior Quality Engineer in Mumbai to own the quality strategy for AI-powered applications. You will lead automation across UI, API, and end-to-end tests using Playwright with Python, Behave, and Selenium where needed.

The role requires 9+ years in QA, strong API testing, and experience with AI/LLM testing, CI/CD, and agile delivery. You will mentor QA engineers and collaborate with cross-functional teams to ensure high-quality releases.

Qualifications

• 9+ years of experience in Quality Engineering and Test Automation.
• Strong hands-on experience in Playwright with Python.
• Experience in building scalable automation frameworks using Python, Pytest, and Behave.
• Experience testing AI, Generative AI, or LLM-powered applications.
• Strong API testing experience using Playwright, Postman, Requests, or REST-assured equivalents.
• Experience with BDD, Git, CI/CD, and Agile delivery practices.
• Knowledge of AI evaluation, prompt testing, RAG validation, hallucination testing, and Responsible AI.
• Strong problem-solving, mentoring, communication, and stakeholder-management skills.

Responsibilities

• Own the end-to-end test strategy, automation roadmap, quality gates, and release-readiness criteria.
• Design and maintain automation frameworks using Playwright with Python and BDD using Behave.
• Automate UI, API, integration, regression, and end-to-end testing.
• Support UAT automation and maintain existing regression suites using Selenium, where required.
• Validate AI/LLM systems for accuracy, relevance, hallucination, bias, safety, context retention, and fallback handling.
• Test prompt injection, harmful prompts, edge cases, RAG workflows, and AI agent integrations.
• Build LLM evaluation frameworks, benchmark datasets, scorecards, and regression suites.
• Integrate automated tests into CI/CD pipelines and implement continuous quality gates.
• Conduct basic performance testing covering p50/p95 response times, throughput, load, and SLA compliance.
• Debug issues using logs, traces, database validation, and backend monitoring.
• Define quality dashboards and ensure traceability, auditability, and compliance.
• Mentor QA engineers and collaborate with Product, Engineering, Data Science, Security, DevOps, and business teams.

Skills

Playwright with Python

BDD with Behave

API automation

AI/LLM testing

Selenium

CI/CD

Mentoring

Test automation strategy

Education

Bachelor's degree in CS/IT

Tools

Python

Pytest

Behave

Postman

Requests

Git

Job description

The Team

Deloitte’s Technology & Transformation practice can help you uncover and unlock the value buried deep inside vast amounts of data. Our global network provides strategic guidance and implementation services to help companies manage data from disparate sources and convert it into accurate, actionable information that can support fact-driven decision-making and generate an insight-driven advantage. Our practice addresses the continuum of opportunities in business intelligence & visualization, data management, performance management and next-generation analytics and technologies, including big data, cloud, cognitive and machine learning.

Your work profile

We are looking for a Senior Quality Engineer with strong expertise in Playwright with Python, API automation, and AI/LLM testing. The role will own the quality strategy, develop scalable automation frameworks, and ensure the accuracy, safety, reliability, performance, and compliance of AI-powered applications.

Key skills required

• Own the end-to-end test strategy, automation roadmap, quality gates, and release-readiness criteria.
• Design and maintain automation frameworks using Playwright with Python and BDD using Behave.
• Automate UI, API, integration, regression, and end-to-end testing.
• Support UAT automation and maintain existing regression suites using Selenium, where required.
• Validate AI/LLM systems for accuracy, relevance, hallucination, bias, safety, context retention, and fallback handling.
• Test prompt injection, harmful prompts, edge cases, RAG workflows, and AI agent integrations.
• Build LLM evaluation frameworks, benchmark datasets, scorecards, and regression suites.
• Integrate automated tests into CI/CD pipelines and implement continuous quality gates.
• Conduct basic performance testing covering p50/p95 response times, throughput, load, and SLA compliance.
• Debug issues using logs, traces, database validation, and backend monitoring.
• Define quality dashboards and ensure traceability, auditability, and compliance.
• Mentor QA engineers and collaborate with Product, Engineering, Data Science, Security, DevOps, and business teams.

Required Skills

• 9+ years of experience in Quality Engineering and Test Automation.
• Strong hands-on expertise in Playwright with Python.
• Experience in building scalable automation frameworks using Python, Pytest, and Behave.
• Experience testing AI, Generative AI, or LLM-powered applications.
• Strong API testing experience using Playwright, Postman, Requests, or REST-assured equivalents.
• Experience with BDD, Git, CI/CD, and Agile delivery practices.
• Knowledge of AI evaluation, prompt testing, RAG validation, hallucination testing, and Responsible AI.
• Strong problem-solving, mentoring, communication, and stakeholder-management skills.

Education

Bachelor’s degree in computer science, Information Technology, or related field.