AI Senior QA Hiver

Earnmodes; Bengaluru; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Hiver is seeking an AI Senior QA Engineer to ensure high product quality for AI features across agents and classifiers. You will own testing strategy, automate tests for UI, API, and backend services, and collaborate with engineering and product teams to ship reliable software.

You will design evals, maintain golden datasets, and drive quality left in a fast-paced SaaS environment. This hands-on role combines debugging, automation, and release readiness to reduce risk.

Qualifications

• 4+ years of software quality engineering experience.
• Hands-on QA for AI features (agents, copilots, RAG, classification).
• Experience building golden datasets from real customer data.

Responsibilities

• Own end-to-end quality for assigned product areas/features.
• Design, review, and execute test strategies (functional, regression, integration).
• Lead feature-level release sign-offs with eng and product.
• Debug UI, backend, and APIs to identify root causes.

Skills

QA automation

Python/TypeScript

CI/CD

AIQA

Release quality ownership

Tools

pytest

Playwright

Langfuse

LangSmith

Braintrust

promptfoo

Job description

Job Description

About us:

Hiver is a modern, AI-driven customer service platform used by companies across healthcare, finance, logistics, education, and technology. We help teams deliver fast, human support across email, chat, phone, WhatsApp, and more — without the complexity of legacy helpdesks.

We’re a challenger brand in a category dominated by over-engineered tools. We build software that is simple, powerful, and genuinely helpful, and we operate internally with that same philosophy. If you want meaningful ownership, thoughtful teammates, and work that ships, Hiver is a great place to do it.

Opportunity:

We are looking for a AI Senior QA Engineer who will play a critical role in ensuring high product quality, reducing release risks, and scaling automation across teams. This is a hands-on, high-ownership individual contributor role where you will influence quality practices, collaborate deeply with engineers, and act as a quality champion within your product area.This role goes beyond test execution—you will help define how quality is built into the system while still being deeply involved in automation, debugging, and release rreadiness.

Key Responsibilities

What you will do?

Quality Ownership & Execution

• Own end-to-end quality for assigned product areas/features.
• Design, review, and execute comprehensive test strategies (functional, regression, integration).
• Lead feature-level release sign-offs in collaboration with engineering and product.
• Actively debug failures across UI, backend, and APIs to identify root causes.
• Work closely with Customer Support to reproduce, analyze, and prevent customer-reported issues.

Automation & Engineering Excellence

• Build, extend, and maintain automated test suites for UI, API, and backend services.
• Improve test reliability by identifying and fixing flaky tests.
• Drive reduction in manual regression time through automation ROI.
• Integrate automated tests into CI/CD pipelines and ensure fast feedback cycles.
• Collaborate with developers to shift quality left (testability, better coverage, early validation).

Key Requirements

What we’re looking for?

• 4+ years of experience in software quality engineering.
• Hands-on QA experience on customer-facing AI features such as agents, copilots, RAG or classification.
• A track record of building and maintaining golden datasets from real customer data.
• Experience designing evals: rubrics, LLM-as-judge, and regression runs on every prompt or model change.
• Strong coded automation (Python or TypeScript, pytest or Playwright, CI) that replaces manual QA.
• The judgement to own the release bar for a non-deterministic product.

Good to Have skills?

• Experience with eval and observability tools such as Langfuse, LangSmith, Braintrust, Ragas, DeepEval or promptfoo.
• Red-teaming and safety testing: prompt injection, jailbreaks, PII leakage and bias.
• Background in B2B SaaS, customer support or helpdesk products.
• Synthetic test data or conversation generation, and using LLMs to generate test cases.
• Production quality monitoring, including sampling live traffic, human review loops and quality dashboards.
• Working knowledge of how LLM features are built (prompts, RAG, tool calling), so failures can be traced to a cause.

Source: Company Career page