QA AI Automation Engineer

NStarX Inc.; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

NStarX Inc. in Hyderabad is seeking a QA AI Automation Engineer to own end-to-end test automation for an enterprise AI platform. Design and execute functional, integration, and LLM-specific regression suites to ensure precision and reliability.

Lead performance/load testing, orchestrate release validation, and integrate testing into CI/CD workflows with a platform team handling security scanning. You will collaborate with teams to produce validation evidence for critical milestones and build

Qualifications

• Backend QA and automation for complex backend services or distributed systems.
• Expert-level Python testing skills with API-focused automation.
• Ownership of regression strategies for highly integrated data-driven systems.
• Experience embedding tests into CI/CD pipelines and release workflows.

Responsibilities

• Automation Suite Ownership: Build, maintain, and scale functional, integration, and regression test suites.
• Performance & Load Testing: Design and execute high-scale load tests to ensure resilience.
• CI/CD Integration: Embed automated tests into CI/CD pipelines for seamless workflows.
• Accuracy & Validation Design: Develop validation strategies with precision/recall style metrics.
• Release Management: Provide validation evidence packages for production milestones.

Skills

Backend automation

Python testing

API testing

Integration testing

Performance testing

CI/CD

CI/CD pipelines

Test strategy

Statistical test design

Tools

pytest

k6

Locust

JMeter

Job description

QA AI Automation Engineer

Job Description:

We are seeking a meticulous and forward-thinking QA AI Automation Engineer to own end-to-end test automation for our enterprise AI platform. In this role, you will design and execute functional, integration, and LLM-specific regression suites to ensure platform reliability and precision. You will also lead performance/load testing, orchestrate release management, and compile validation evidence for critical delivery milestones, working alongside a platform team that handles security scanning and gate enforcement.

Key Responsibilities:

• Automation Suite Ownership: Build, maintain, and scale comprehensive functional, integration, and regression test suites for backend and distributed systems.


• Performance & Load Testing: Design and execute high-scale load and performance testing to ensure system resilience under enterprise workloads.


• CI/CD Integration: Embed automated test execution directly into CI/CD pipelines to support a seamless, continuous integration workflow.


• Accuracy & Validation Design: Develop advanced validation strategies (such as precision/recall-style test designs) tailored for accuracy-critical detection systems.


• Release Management: Oversee release management processes across sprint-based delivery cycles, providing necessary validation evidence packages for production milestones.

MINIMUM REQUIREMENTS

• Backend Automation Experience: Proven background in QA and test automation targeting complex backend services or distributed systems.


• Python Test Frameworks: Expert-level mastery of Python test frameworks (specifically pytest or equivalent) with deep experience in API-level automated testing.


• Strategic Integration Testing: Demonstrated ownership of test strategies and regression suites for highly integrated, data-dependent systems.


• Performance Testing at Scale: Hands-on experience with load and performance testing tools such as k6, Locust, JMeter, or equivalent frameworks.


• CI-Driven Development: Experience managing and configuring automated tests to run natively as steps within continuous integration pipelines.


• Sprint-Based Release Management: Solid understanding of release coordination across fast-paced, sprint-based release structures.


• Statistical Test Design: Experience designing evaluation frameworks for data or detection systems using precision/recall and similar statistical validation metrics.

GOOD TO HAVE

• LLM & Agent Evaluation: Experience implementing LLM/agent evaluation frameworks and regression suites using golden data sets or reference workloads.


• DLP / PII Testing: Background in validating Data Loss Prevention (DLP) or PII detection engines, specifically optimizing for zero-false-negative benchmarks.


• Cloud Environments: Familiarity running tests inside Kubernetes-based environments or participating directly in Disaster Recovery (DR) simulations.


• NFR Verification: Experience pulling together formal non-functional requirement verification metrics and compiling structured acceptance evidence packages for enterprise clients.