QA Tester – AI Application

NexTurn Inc.; Hyderabad; On-site; INR 600,000 - 900,000; Full time

Job summary

NexTurn Inc. is seeking a QA/Test Engineer to drive comprehensive testing across manual, functional, API and integration layers for AI-assisted workflows.

You will validate REST APIs, WebSockets and event streams, with SQL data checks and Databricks-based pipelines in regulated healthcare contexts. You will document test plans, execute scenarios, report defects, and collaborate with DevOps/MLOps to ensure release readiness and observability.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
• For a QA/Testing role, hands-on experience across manual, functional, integration, regression and API testing is essential.
• Proficiency with API testing tools such as Postman, REST Assured, SoapUI or Swagger.
• Experience validating REST APIs, WebSockets, event streams and asynchronous workflows.
• Good SQL knowledge for data validation is expected.
• Familiarity with Databricks, Delta tables, data pipelines or cloud data platforms is a plus.
• Knowledge of RAG, LLMs, embeddings, semantic search, knowledge graphs and AI testing.
• Ability to validate AI outputs against grounding, business rules and traceability.
• Experience with test management and defect tracking tools like Jira or Azure DevOps.
• Strong analytical, communication and documentation skills.
• Exposure to healthcare/pharma data workflows, regulatorily influenced environments is a plus.

Responsibilities

• Prepare test strategy, scenarios, cases, data and execution reports for the application.
• Validate end-to-end operator workflows across dashboards and learning consoles.
• Test operator actions like acknowledge, resolve, accelerate and re-evaluate.
• Validate API and live event-stream behavior for real-time updates.
• Test exception handling, queue health metrics and re-evaluation events.
• Validate EDI/X12 parsing and mapping to internal models.
• Test deterministic and stateful decision workflows with tool executions.
• Verify resolution tools such as price checks, drug lookups and document checks.
• Assess AI model outputs, prompts, agent workflows and retrieval quality.
• Ensure observability metrics, auditability and traceability in decisions.
• Log defects with reproducible steps and collaborate for release readiness.

Skills

Manual testing

Functional testing

Integration testing

Regression testing

API testing

API testing tools

REST APIs

WebSockets

Event streams

Asynchronous workflows

SQL basics

Databricks

Delta tables

Data pipelines

Cloud platforms

RAG

LLMs

Embeddings

Semantic search

Knowledge graphs

AI testing

AI validation

Rule-based workflows

Exception workflows

State machines

Case management

Auth

Authorization

RBAC

Data privacy

Audit validation

Jira

Azure DevOps

Analytical skills

Communication

Documentation

Azure Databricks

Unity Catalog

Mosaic AI

MLflow

EDI/X12

Healthcare data

Vector search

Graph databases

Context graphs

Triple stores

Decision trace stores

PyTest

Python

Model validation

Prompt changes

Agent workflows

Retrieval quality

FDA data

DEA validation

NADAC pricing

EPCIS

Serialization

CI/CD

Smoke testing

Release certification

Sanity testing

Grafana

Prometheus

OpenTelemetry

Azure Monitor

Log Analytics

Education

Bachelor's or Master's degree in Computer Science or IT

Tools

Postman

REST Assured

SoapUI

Swagger

Jira

Azure DevOps

Python

Job description

• Strong experience in manual, functional, integration, regression, and API testing.
• Experience with API testing using Postman, REST Assured, SoapUI, Swagger, or similar tools.
• Experience testing REST APIs, WebSockets, event streams, and asynchronous workflows.
• Basic SQL knowledge for data validation.
• Understanding Databricks, Delta tables, data pipelines, or cloud-based data platforms.
• Familiarity with RAG, LLMs, embeddings, semantic search, knowledge graphs, and AI application testing.
• Ability to validate AI responses against source grounding, expected business rules, and trace evidence.
• Experience testing rule-based workflows, exception workflows, state machines, or case management systems.
• Understanding authentication, authorization, role-based access, data privacy, and audit validation.
• Experience with test management and defect tracking tools such as Jira, Azure DevOps, or similar.
• Good analytical, communication, and documentation skills.
• Experience testing applications on Azure Databricks, Unity Catalog, Mosaic AI, or MLflow.
• Exposure to EDI/X12, healthcare transactions, pharmacy, payer, provider, claims, pricing, or drug data workflows.
• Exposure to vector search, graph databases, context graphs, triple stores, or decision trace stores.
• Experience with automated API testing and Python-based test automation frameworks such as PyTest.
• Experience validating model outputs, prompt changes, agent workflows, and retrieval of quality.
• Familiarity with FDA drug data, DEA registry validation, NADAC/GPO pricing, EPCIS, VRS, or serialization workflows.
• Familiarity with CI/CD validation, smoke testing, release certification, and production sanity testing.
• Knowledge of observability tools such as Grafana, Prometheus, OpenTelemetry, Azure Monitor, or Log Analytics.
• Understanding healthcare, pharma, or regulated enterprise environments is a plus.

Qualifications: Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

Job Description:

• Prepare test strategy, test scenarios, test cases, test data, and test execution reports for the Application.
• Validate end-to-end operator workflows across dashboard, exception workspace, context inspection, learning console, and local client state behavior.
• Test operator actions such as acknowledge, resolve, accelerate, annotate, re-evaluate, and case state transitions.
• Validate transactional request/response APIs and live event-stream behavior for real-time updates.
• Test exception arrival, agent progress updates, queue health metrics, and re-evaluation events.
• Validate EDI/X12 parsing, exception normalization, and mapping to the internal exception model.
• Test deterministic exception handling through L1 reference data lookups and L2 policy-based routing.
• Validate L3 agentic resolution workflows, including bounded tool execution and fallback behavior.
• Test resolution tools such as drug lookup, pricing check, serial verification, DEA check, and document lookup.
• Verify Graph RAG and Document RAG results for relevance, accuracy, grounding, source traceability, and response consistency.
• Validate context retrieval from context graph, reference data, decision trace store, policies, documentation, and regulatory guidance.
• Test decision trace generation for both deterministic and agentic decisions.
• Validate auditability, explainability, trace completeness, and decision lineage.
• Test feedback capture from operator resolutions and learning-loop workflows.
• Validate pattern mining outputs, candidate rule generation, routing optimization results, validation of sandbox results, and promotion of gate approvals.
• Perform regression testing after model, prompt, rule, routing, ingestion, retrieval, or external-source changes.
• Validate Unity Catalog permissions, data governance, access controls, and restricted data visibility.
• Conduct negative testing, boundary testing, failure scenario testing, retry testing, idempotency testing, and security validation support.
• Validate observability metrics such as queue health, exception volume, agent latency, retrieval quality, provider errors, token usage, ingestion lag, and decision trace completeness.
• Log defects with clear reproduction steps, expected results, actual results, screenshots/logs, test data references, and severity.
• Collaborate with developers, DevOps/MLOps, architects, business users, and operators for defect triage and release readiness.