Associate Lead - Testing (QA + MLOps)

Quantiphi Analytics Solutions Private Limited; Bengaluru; Hybrid; INR 2,800,000 - 5,200,000; Full time

Job summary

Quantiphi Analytics Solutions Private Limited seeks an experienced Lead/Associate Lead - QA + MLOps & GenAI to shape AI testing across AWS services, including SageMaker, Bedrock, and Lambda. Hybrid work in Mumbai/Bangalore, 10+ years in QA/AI testing required, with strong Python and CI/CD skills.

The role covers model validation, data drift detection, RAG validation, and safety/testing for GenAI, with leadership of QA teams and client engagements.

Qualifications

• 10+ years in Quality Engineering with AI/ML focus.
• Experience testing ML models (NLP preferred).
• Hands-on validation of LLM-based apps.

Responsibilities

• Define AI testing strategy for AWS-based AI systems.
• Validate end-to-end ML lifecycle: data, training, deployment.
• Implement CI/CD gates for ML pipelines with AWS DevOps tools.

Skills

QA Testing

MLOps

GenAI Testing

AWS

SageMaker

Bedrock

Python

CI/CD

Tools

TensorFlow

PyTorch

LangChain

Job description

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Role: Lead/Associate Lead - QA + MLOps & Genitive AI Experience: 10+ years Location: Mumbai/Bangalore (Hybrid)

Key Responsibilities

AI/ML & GenAI Testing Strategy (AWS Ecosystem) Define testing approaches for AI systems built on AWS services such as: Amazon SageMaker Amazon Bedrock AWS Lambda Amazon API Gateway Amazon Kinesis AWS Glue Amazon S3 Amazon CloudWatch Design validation frameworks covering: Model accuracy & performance validation Data drift & concept drift detection Hallucination detection for LLMs Prompt robustness testing RAG validation (retrieval accuracy + grounding) Bias & fairness validation Safety & toxicity testing MLOps Quality Engineering (AWS-Centric) Validate the end-to-end ML lifecycle including: Data ingestion & feature pipelines Model training & hyperparameter tuning Model versioning & registry Deployment validation Canary & blue/green release validation Work with AWS-native services such as: SageMaker Pipelines SageMaker Model Monitor SageMaker Feature Store Bedrock model evaluation workflows CloudWatch-based observability Implement CI/CD quality gates for ML pipelines integrated with AWS DevOps tools. GenAI & Agentic AI Testing Define quality engineering approaches for: LLM-based applications using Amazon Bedrock Prompt engineering validation Multi-agent orchestration testing Chatbot & Voice bot conversational testing Intent classification validation Conversation drift & fallback validation API contract validation for LLM integrations Build reusable evaluation harnesses for: BLEU / ROUGE scoring Embedding similarity scoring Response consistency Safety scoring frameworks Framework & Capability Development Design reusable AI testing accelerators Create AWS-aligned AI test automation frameworks (Python-first) Develop synthetic data generation strategies Establish AI quality scorecards Build an internal AI QA Center of Excellence Client Engagement & Leadership Lead AI/ML quality strategy workshops Perform AI risk & readiness assessments Present quality architecture to CXOs Drive QA transformation programs Mentor QA teams on AWS-based AI testing Own delivery for AI testing engagements end-to-end

Must have skills

Testing Expertise 8-12+ years in Quality Engineering Strong test strategy, automation & governance experience Experience leading QA transformation initiatives Experience building frameworks from scratch AI/ML & GenAI Expertise Deep understanding of ML lifecycle Experience testing ML models (NLP preferred) Hands-on experience validating LLM applications Strong understanding of: Prompt engineering RAG architecture Embeddings Bias & explainability AWS AI/ML Expertise Hands-on experience with: Amazon SageMaker (training, deployment, monitoring) Amazon Bedrock (LLM integration & evaluation) S3-based data pipelines AWS IAM (security validation) CloudWatch monitoring Lambda & API Gateway integrations AWS CI/CD (CodePipeline / CodeBuild preferred) Understanding of: Infrastructure as Code (Terraform / CloudFormation) Observability in AI systems Cost monitoring for ML workloads Technical Skills Python (mandatory) Experience with ML libraries (Scikit-learn, TensorFlow, PyTorch) Experience with LLM frameworks (LangChain, etc.) API & automation testing frameworks Git-based workflows Leadership & Communication Strong client-facing communication Experience leading QA teams Ability to create strategy decks & solution proposals Strong stakeholder management

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

Thank you for your interest in Quantiphi! We are a team that dreams together and collaborates to ensure success. We are currently looking for exceptional individuals who can join us and contribute to a fun, diverse and hybrid work culture. As a part of the Quantiphi family, you will get ample opportunities to learn, grow and interact with colleagues from varied experience and backgrounds around the globe. Quantiphi is an award-winning AI-first digital engineering company driven by the desire to reimagine and realize transformational opportunities at the heart of business. We solve the toughest and most complex business problems with the latest and cutting-edge technologies and to make this happen, we have a vibrant, diverse and talented set of professionals we proudly refer to as the Quantiphi family. Our culture is built on transparency, diversity, integrity, learning and growth. We are committed to provide our teams with an environment that helps them to learn, grow and flourish in their professional as well as personal lives.