Senior AI Product Tester

Venxr; Pune District; On-site; INR 1,500,000 - 2,500,000; Full time

Benefits offered by this job

Small team environment

Flat hierarchy

Direct client exposure

Pioneering AI testing

Learning budget

Job summary

Venxr is hiring a Senior AI Product Tester to ensure zero broken AI products reach clients, focusing on QA automation powered by Claude Code for UI, API, and conversational flows. You will build evaluation sets, validate data against ad platforms, and maintain a robust regression suite while working with WhatsApp bot conversations in Hinglish.

You’ll log detailed bugs daily, write test cases from acceptance criteria, and run a daily sanity check across all products.

Qualifications

• 3–5 years in software QA with at least 2 years building CI-integrated automation.
• Hands-on experience using Claude Code to write, debug, and maintain test suites.

Responsibilities

• Own quality across AdsGPT and WhatsApp bot with daily QA practices starting at 10am.
• Write and review test cases from acceptance criteria with PM.
• Run a daily 30-minute sanity check across products and flag issues before stand-up.
• Build QA automation using Claude Code for UI, API, and conversational flows, integrated into CI.
• Log 5–10 actionable bugs daily with clear repro steps and evidence.
• Create evaluation sets for AI features to measure prompt/model improvements.
• Validate dashboards against Google Ads and Meta down to the rupee.
• Maintain regression suite by fixing flaky tests and closing gaps.
• Test non-deterministic outputs using structure/intent assertions and LLM-as-judge.
• Test bulk uploads for safe failure reporting and precise error messages.
• Verify automated rules (spend alerts) fire correctly or stay silent as needed.
• Test WhatsApp bot conversations including Hinglish, short replies, and prompt injections.
• Validate live client reports for data accuracy against ad platforms.

Skills

QA automation

CI integration

Claude Code

Playwright

Cypress

Selenium

Python

JavaScript/TypeScript

SQL

Bug reporting

LLM evaluation

Tools

Claude Code

Playwright

Cypress

Selenium

Postman

pytest

REST Assured

GitHub Actions

Docker

k6

Job description

About The Opportunity

Venxr is hiring a Senior AI Product Tester to ensure zero broken AI products reach clients, especially those managing real ad budgets for major Indian brands like Haier, Orient Electric, and Suzuki. The role centers on QA automation powered by Claude, testing products that act on data — launching campaigns, moving budgets, and conversing with customers via WhatsApp.

Role & Responsibilities

• Own quality across AdsGPT, and the WhatsApp bot — not as a gate at sprint end, but as a daily habit starting at 10am and embedded in every story shipped.
• Write test cases for every user story from acceptance criteria, and review them with the PM before development finishes.
• Run a 30-minute morning sanity check daily across all products — logins, key flows, ad-platform syncs, bot replies — and flag red issues before stand-up.
• Build and own QA automation using Claude Code for UI, API, and conversational flows, wired into CI to run on every deploy.
• Log 5–10 actionable bugs daily: clear repro steps, expected vs actual, severity, and evidence — so developers can fix without follow-up.
• Build evaluation sets for AI features — golden prompts, edge cases, adversarial inputs — to measure if new prompts or models improve outcomes.
• Validate numbers against source systems — dashboards must match Google Ads and Meta down to the rupee.
• Maintain the regression suite: fix flaky tests, delete useless ones, and identify coverage gaps.
• Test non-deterministic AI outputs using structural/intent assertions, LLM-as-judge with rubrics, and human validation when needed.
• Ensure bulk uploads (e.g., Excel-based ad campaigns) fail safely and report precise errors for malformed inputs.
• Verify automated rules (e.g., spend alerts) fire correctly and stay silent when they shouldn’t.
• Test WhatsApp bot conversations — Hinglish, one-word replies, angry customers, off-topic questions, prompt injection.
• Validate live, self-updating client reports to ensure data accuracy against ad platforms.

Skills & Qualifications

Must-Have

• 3–5 years in software QA, with at least 2 years building CI-integrated automation.
• Hands-on experience using Claude Code (or similar coding agent) to write, debug, and maintain test suites — must show a working suite built this way.
• Proficient with Playwright, Cypress, or Selenium for UI testing, and Postman, pytest, or REST Assured for API testing.
• Comfortable reading and testing code in Python or JavaScript/TypeScript.
• Skilled at testing non-deterministic outputs — assertions on structure/intent, LLM-as-judge with rubrics, and knowing when human judgment is needed.
• Ability to write clear, concise, and actionable bug reports and test cases — no padding, just specifics.
• Working knowledge of SQL to validate data directly in the database, not just UI.

Preferred

• Experience testing chatbots or WhatsApp Business API flows.
• Exposure to Google Ads, Meta Ads, or ad-tech platforms.
• Familiarity with MCP servers (e.g., Playwright MCP), GitHub Actions, or Docker.
• Basic load testing experience with k6, Locust, or JMeter.

Benefits & Culture Highlights

• Small, fast team — bugs you find get fixed in days, not quarters.
• Build QA automation from the ground up — you choose tools, structure, and what to automate first.
• Flat hierarchy — no layers between you and release decisions.
• Direct client exposure — see how brands use your work and learn the ad industry.
• Work at the edge of AI testing — help write the playbook as you go.
• Offsites, learning budget, and an open culture for asking questions.

Skills: typescript,rest assured,k6,claude code,python,selenium,sql,github actions,pytest,javascript,cypress,playwright