AI Senior QA

Hiver; Bengaluru; On-site; INR 1,500,000 - 2,200,000; Full time

Job summary

Hiver, a modern AI-driven customer service platform, is seeking an AI Senior QA Engineer in Bangalore. This hands-on, high-ownership role blends test strategy, automation, and release readiness with deep collaboration with engineering and product teams.

You will own end-to-end quality for key product areas, design evals for AI features, build automated UI/API test suites, and drive faster feedback through CI/CD integration and robust defect analysis.

Qualifications

• 4+ years of software quality engineering experience.
• Hands-on QA for customer-facing AI features such as agents, copilots, RAG or classification.
• Proven track record of building golden datasets from real customer data.
• Experience designing evals: rubrics, LLM-as-judge, and regression runs on every prompt or model change.

Responsibilities

• Own end-to-end quality for assigned product areas/features.
• Design, review, and execute comprehensive test strategies (functional, regression, integration).
• Lead feature-level release sign-offs in collaboration with engineering and product.
• Actively debug failures across UI, backend, and APIs to identify root causes.
• Work closely with Customer Support to reproduce, analyze, and prevent customer-reported issues.
• Build, extend, and maintain automated test suites for UI, API, and backend services.
• Improve test reliability by identifying and fixing flaky tests.
• Drive reduction in manual regression time through automation ROI.
• Integrate automated tests into CI/CD pipelines and ensure fast feedback cycles.
• Collaborate with developers to shift quality left (testability, better coverage, early validation).

Skills

QA automation

Python/TypeScript

CI/CD

LLM evaluation

Test strategy design

Release ownership

Tools

pytest

Playwright

Job description

AI Senior QA

Department: AI

Employment Type: Full Time

Location: Bangalore - India

Reporting To: Anurag Maherchandani

Description

About us:

Hiver is a modern, AI-driven customer service platform used by companies across healthcare, finance, logistics, education, and technology. We help teams deliver fast, human support across email, chat, phone, WhatsApp, and more - without the complexity of legacy helpdesks.

We're a challenger brand in a category dominated by over-engineered tools. We build software that is simple, powerful, and genuinely helpful, and we operate internally with that same philosophy. If you want meaningful ownership, thoughtful teammates, and work that ships, Hiver is a great place to do it.

Opportunity:

We are looking for a AI Senior QA Engineer who will play a critical role in ensuring high product quality, reducing release risks, and scaling automation across teams. This is a hands-on, high-ownership individual contributor role where you will influence quality practices, collaborate deeply with engineers, and act as a quality champion within your product area. This role goes beyond test execution - you will help define how quality is built into the system while still being deeply involved in automation, debugging, and release rreadiness.

Key Responsibilities

What you will do?

• Quality Ownership & Execution
• Own end-to-end quality for assigned product areas/features.
• Design, review, and execute comprehensive test strategies (functional, regression, integration).
• Lead feature-level release sign-offs in collaboration with engineering and product.
• Actively debug failures across UI, backend, and APIs to identify root causes.
• Work closely with Customer Support to reproduce, analyze, and prevent customer-reported issues.
• Automation & Engineering Excellence
• Build, extend, and maintain automated test suites for UI, API, and backend services.
• Improve test reliability by identifying and fixing flaky tests.
• Drive reduction in manual regression time through automation ROI.
• Integrate automated tests into CI/CD pipelines and ensure fast feedback cycles.
• Collaborate with developers to shift quality left (testability, better coverage, early validation).

Key Requirements

What we’re looking for?

• 4+ years of experience in software quality engineering.
• Hands-on QA experience on customer-facing AI features such as agents, copilots, RAG or classification.
• A track record of building and maintaining golden datasets from real customer data.
• Experience designing evals: rubrics, LLM-as-judge, and regression runs on every prompt or model change.
• Strong coded automation (Python or TypeScript, pytest or Playwright, CI) that replaces manual QA.
• The judgement to own the release bar for a non-deterministic product.

Good to Have skills?

• Experience with eval and observability tools such as Langfuse, LangSmith, Braintrust, Ragas, DeepEval or promptfoo.
• Red-teaming and safety testing: prompt injection, jailbreaks, PII leakage and bias.
• Background in B2B SaaS, customer support or helpdesk products.
• Synthetic test data or conversation generation, and using LLMs to generate test cases.
• Production quality monitoring, including sampling live traffic, human review loops and quality dashboards.
• Working knowledge of how LLM features are built (prompts, RAG, tool calling), so failures can be traced to a cause.
• I kept these out of the screening gates, so they only help rank candidates who already pass. I can add them to the skill as tie-breakers if you want.