Senior AI QA Engineer

Get Well; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Get Well is seeking a Senior AI QA Engineer in Bengaluru. This role involves developing validation frameworks for clinical insights, requiring extensive experience in Quality Assurance and a strong proficiency in Python. Responsibilities include automating model grading, executing data integrity tests, and collaborating with Data Scientists. Familiarity with AI/ML validation metrics and tools like LangSmith and PyTest is essential. Join a dynamic team committed to transforming patient care through innovative AI solutions.

Qualifications

• 6+ years of technical experience in Quality Assurance.
• Expert‑level skills in Python for building custom test scripts.
• Proven experience testing model outputs using diverse metrics.

Responsibilities

• Design and implement structured testing strategies.
• Develop systems to evaluate clinical pipeline outputs.
• Collaborate with Data Scientists and Product Managers.

Skills

Quality Assurance

Python

AI/ML Validation

API Testing

Strategic Thinking

Tools

LangSmith

PyTest

Postman

Job description

Opportunity

As a Senior AI QA Engineer for our Precision Patient Care Pipeline, you will go beyond traditional functional testing. You will be responsible for building the framework that ensures our clinical insights are accurate, safe, and reliable. This role requires a unique blend of high-level software testing and data engineering to validate complex, non‑deterministic medical outputs using a hybrid of automated grading methodologies.

Title: Senior AI QA Engineer, Location: Bengaluru (Bangalore)

Key Responsibilities

• Architect Multi‑Layered Validation Frameworks: Design and implement structured testing strategies that combine deterministic checks, semantic similarity metrics, and model‑based evaluations.
• Automated Model Grading: Develop systems to evaluate clinical pipeline outputs for faithfulness, safety, and hallucination detection using various automated scoring techniques (e.g., BERTScore, ROUGE, or custom heuristics).
• Vibe‑Driven Development: Leverage agentic AI tools to rapidly prototype complex test harnesses, "red‑team" clinical logic, and build internal validation utilities at high velocity.
• Data Pipeline Integrity: Execute integration and regression tests for data‑heavy backend processes, ensuring medical data remains consistent from ingestion to insight generation.
• Collaborative Strategy: Work closely with Data Scientists and Product Managers to define "Ground Truth" datasets and clinical evaluation rubrics.
• Root Cause Analysis: Deep‑dive into complex system failures to identify whether issues stem from code logic, data drift, or model behavior.

Requirements

• 6+ years of technical experience in Quality Assurance, with a strong focus on system architecture and backend data validation.
• Advanced Python Proficiency: Expert‑level skills in Python for building custom test scripts and working within AI/ML ecosystems.
• AI/ML Validation Experience: Proven experience testing model outputs using diverse metrics (e.g., Semantic Similarity, NLP metrics, and custom scoring algorithms). Familiarity with tools like LangSmith, Arize Phoenix, or MLflow.
• Vibe Coding Mastery: Proficiency with AI‑native development tools such as Cursor, Windsurf, Claude Code.
• Modern Automation Stack: Proficiency with frameworks like PyTest, and experience with traditional tools (Cypress, Playwright, or Selenium) for end‑to‑end flows.
• API & Data Testing: Solid understanding of API testing (Postman/RestAssured) and database validation (SQL/NoSQL).
• Strategic Thinking: Experience serving as a Subject Matter Expert (SME) on test process development for large‑scale systems.

Preferred (Good To Have)

• Clinical Domain Knowledge: Familiarity with healthcare data standards such as FHIR, HL7, or SNOMED CT.
• Healthcare Compliance: Understanding of HIPAA regulations and the nuances of handling ePHI/PHI.
• Statistical Foundation: Understanding of sensitivity, specificity, and F1‑scores in a clinical context.

GW RhythmX is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age or veteran status.