AI Senior QA

Hiver, Inc.; Bengaluru; On-site; INR 2,200,000 - 3,400,000; Full time

Job summary

Hiver is hiring an AI Senior QA engineer in Bangalore, India. This hands-on role offers high ownership to drive product quality for AI features such as agents and classifications, reducing release risks.

You will define test strategies, automate UI/API tests, and collaborate with engineers and support teams. We seek 4+ years in software QA with strong automation in Python or TypeScript, using Playwright or pytest.

Qualifications

• 4+ years of software quality engineering experience.
• Hands-on QA for customer-facing AI features (agents, copilots, RAG).
• Experience building golden datasets from real customer data.
• Strong automation in Python or TypeScript with pytest or Playwright.
• Ability to own release bar for a non-deterministic product.

Responsibilities

• Own end-to-end quality for assigned product areas.
• Design and execute test strategies (functional, regression, integration).
• Lead feature-level release sign-offs with engineering and product.
• Develop and maintain automated test suites (UI, API, backend).
• Integrate tests into CI/CD and improve feedback loops.

Skills

Quality engineering

Automation scripting

Python

TypeScript

CI/CD

Debugging

Test strategy

AI QA

Tools

Playwright

pytest

Job description

AI Senior QA

Department: AI

Employment Type: Full Time

Location: Bangalore - India

Reporting To: Anurag Maherchandani

Description

About us:

Hiver is a modern, AI-driven customer service platform used by companies across healthcare, finance, logistics, education, and technology. We help teams deliver fast, human support across email, chat, phone, WhatsApp, and more — without the complexity of legacy helpdesks.

We’re a challenger brand in a category dominated by over-engineered tools. We build software that is simple, powerful, and genuinely helpful, and we operate internally with that same philosophy. If you want meaningful ownership, thoughtful teammates, and work that ships, Hiver is a great place to do it.

Opportunity:

We are looking for a AI Senior QA Engineer who will play a critical role in ensuring high product quality, reducing release risks, and scaling automation across teams. This is a hands‑on, high-ownership individual contributor role where you will influence quality practices, collaborate deeply with engineers, and act as a quality champion within your product area. This role goes beyond test execution—you will help define how quality is built into the system while still being deeply involved in automation, debugging, and release rreadiness.

Key Responsibilities

What you will do?

• Quality Ownership & Execution
• Own end‑to‑end quality for assigned product areas/features.
• Design, review, and execute comprehensive test strategies (functional, regression, integration).
• Lead feature-level release sign-offs in collaboration with engineering and product.
• Actively debug failures across UI, backend, and APIs to identify root causes.
• Work closely with Customer Support to reproduce, analyze, and prevent customer‑reported issues.
• Automation & Engineering Excellence
• Build, extend, and maintain automated test suites for UI, API, and backend services.
• Improve test reliability by identifying and fixing flaky tests.
• Drive reduction in manual regression time through automation ROI.
• Integrate automated tests into CI/CD pipelines and ensure fast feedback cycles.
• Collaborate with developers to shift quality left (testability, better coverage, early validation).

Key Requirements

What we’re looking for?

• 4+ years of experience in software quality engineering.
• Hands‑on QA experience on customer‑facing AI features such as agents, copilots, RAG or classification.
• A track record of building and maintaining golden datasets from real customer data.
• Experience designing evals: rubrics, LLM-as-judge, and regression runs on every prompt or model change.
• Strong coded automation (Python or TypeScript, pytest or Playwright, CI) that replaces manual QA.
• The judgement to own the release bar for a non-deterministic product.

Good to Have skills?

• Experience with eval and observability tools such as Langfuse, LangSmith, Braintrust, Ragas, DeepEval or promptfoo.
• Red‑teaming and safety testing: prompt injection, jailbreaks, PII leakage and bias.
• Background in B2B SaaS, customer support or helpdesk products.
• Synthetic test data or conversation generation, and using LLMs to generate test cases.
• Production quality monitoring, including sampling live traffic, human review loops and quality dashboards.
• Working knowledge of how LLM features are built (prompts, RAG, tool calling), so failures can be traced to a cause.