AdvaRisk - QA Lead - Automation Testing

Mployee.me; Pune District; On-site; INR 2,500,000 - 3,500,000; Full time

Job summary

Mployee.me is seeking a Senior QA Automation Lead in Pune to own the test automation strategy across UI, API, and backend layers.

You'll lead a QA team of 3+ engineers, collaborate with product, data science/ML, and client services to ensure high-quality releases and robust automation coverage for AI/ML workloads.

Qualifications

• Proven experience building automation frameworks and maintaining them at scale.
• Strong testing fundamentals with hands-on automation across UI/API/backend.
• Experience leading and mentoring a QA team of 3+ engineers.
• Expertise in API testing, including REST, validation and error handling.

Responsibilities

• Own end-to-end test automation strategy and standards for UI, API, backend and integration testing.
• Lead, mentor and manage the QA team; own delivery and quality outcomes.
• Define acceptance criteria with cross-functional teams (engineering, product, data science/ML).
• Create and maintain test plans, data, traceability and release quality reports.
• Triage defects and coordinate prioritization with relevant teams.
• Test AI/ML model integrations end-to-end and validate evaluation metrics.
• Plan and execute performance, load, and stress tests; analyze bottlenecks.

Skills

Automation leadership

Python automation

QA team management

API testing

Performance testing

AI/ML testing

LLM evaluation

SQL testing

CI/CD integration

Front-end testing

Agile/Scrum

Tools

Pytest

Robot Framework

Selenium

Cypress

JMeter

Locust

k6

Job description

Key Roles & Responsibilities

• Own and drive the end-to-end test automation strategy, defining the roadmap, frameworks, tooling and standards for UI, API, backend and integration testing.
• Lead, mentor and manage the QA team, owning delivery, prioritization, test coverage and quality outcomes.
• Work closely with engineering, product, data science/ML and client service teams to clarify requirements and define acceptance criteria.
• Create and maintain test plans, test scenarios, test data, traceability and release-level quality reports.
• Triage defects, assess business and customer impact, and coordinate defect prioritization with relevant teams.
• Test AI/ML model integrations end to end, including request/response contracts, preprocessing and post-processing, timeout/retry behavior, failure handling and fallback paths.
• Build model-focused test suites using representative and edge-case datasets; assess output correctness against agreed reference labels or acceptance criteria, and track metrics such as precision/recall where applicable.
• Validate LLM-enabled workflows for groundedness, consistency, hallucination risks, prompt/input sensitivity and regression when prompts, models or versions change; use defined evaluation criteria rather than subjective checks alone.
• Check model-service latency, throughput, rate limits, availability, version compatibility and behavior under malformed, incomplete or unexpected inputs.
• Perform end-to-end, system integration and regression testing across UI, backend services, APIs, databases and third party/model services.
• Establish performance baselines and report findings with actionable recommendations for capacity and scalability improvements.
• Plan and execute performance, load, stress, spike and endurance tests; analyze throughput, latency, concurrency, resource utilization and bottlenecks.
• Work with developers to reproduce issues, verify fixes and prevent regressions throughout the release cycle.
• Own test-suite execution in non-production environments and integrate automated checks into CI/CD pipelines where appropriate.

Required Skills

• Strong experience testing web applications across UI, backend services, APIs, databases and end-to-end workflows.
• Strong QA fundamentals with substantial hands-on automation experience using Python; able to design, build, maintain and scale automation frameworks rather than only execute existing scripts.
• People leadership experience, including leading, mentoring and driving the output of a team of 3 or more QA engineers.
• Strong API testing and debugging skills, including REST APIs, response/schema validation, authentication, error handling and service-to-service integration.
• Experience with performance-testing tools such as JMeter, Locust, k6 or equivalent, and ability to interpret results and identify bottlenecks.
• Working knowledge of AI/ML model evaluation and integration testing; able to define test datasets, expected outcomes, thresholds and regression checks in collaboration with data science/ML engineers.
• Familiarity with testing LLM-based features, including prompt/version regression, groundedness and response-quality evaluation, structured-output validation and safe failure handling.
• Comfortable validating data and service behavior using SQL, logs, metrics and traces; familiarity with CI/CD-based automated test execution is an advantage.
• Good understanding of front-end technologies and browser-based testing, including JavaScript, HTML and CSS.
• Hands-on experience building and maintaining automation frameworks using tools such as Pytest, Robot Framework, Selenium, Cypress or equivalent; experience with API automation is essential.
• Experience working in Agile/Scrum teams and participating in sprint planning, refinement, reviews and release readiness.
• A strong quality mindset and a track record of promoting automation-first, risk-based testing and continuous quality improvement.
• Clear communication skills and the ability to work across engineering, product, QA and AI/ML teams to define and deliver quality outcomes.

Behavioral Expectations

• Be excited by and have a positive outlook to navigate ambiguity.
• Passion for results and excellence.
• Team player.
• Must be able to get the job done by working collaboratively with others.
• Inquisitive and analytical mind; out-of-the-box thinking.
• Prioritize among competing opportunities, balancing consumer needs with business and product priorities, and clearly articulate the rationale behind product decisions.
• Straightforward and professional.
• Good communicator.
• Maintain high energy and motivate the team.
• A do-it-yourself orientation, consistent with the company's "roll-up-the-sleeves" culture.
• Proactive.

(ref:hirist.tech)