SDET - AI ML Validation Engineer

Quality Ai; Bengaluru; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

Quality Ai is looking for an SDET - AI ML Validation Engineer in Bangalore. This role involves validating AI solutions and chatbots for accuracy and reliability. You will also curate datasets, perform testing using Python and ensure comprehensive API testing.

Candidates should have 7-10 years of experience and proficiency in automation frameworks like Pytest. The position offers a hybrid working environment, integrating modern cloud and CI/CD practices.

Qualifications

• 7-10 years of experience in software development and testing.
• Proficient in AI validation and automation frameworks.
• Experience with REST/GraphQL APIs and cloud services.

Responsibilities

• Validate AI-powered solutions for accuracy and reliability.
• Test chatbots and generative AI features.
• Curate datasets and validate data pipelines.

Skills

AI Validation & Testing

Python

UI & API Testing

Tools

Postman

Jenkins

AWS

Job description

SDET - AI ML Validation Engineer

Date: 1 Jul 2026 | Company: QualityAI | Region: IN

Experience Range: 7-10 years

Job Location: Bangalore with Hybrid

Primary Skill

AI Validation & Testing

Chatbot & RAG Testing • LLM/Prompt Evaluation • Python Validation

AI Solution Testing

• Validates AI-powered solutions including chatbots and generative AI features for accuracy, reliability, and edge cases
• Tests Retrieval-Augmented Generation pipelines for retrieval relevance, grounding, and hallucination handling
• Builds and tests agentic solutions such as MCP and prompting

Test Data & AI Evaluation

Data Quality • AI Eval Metrics • Responsible AI

• Curates and versions golden/evaluation datasets and validates data pipelines for integrity, completeness, and drift across training and inference
• Hands‑on with evaluation frameworks and metrics such as RAGAS, DeepEval, and LLM-as-judge, covering groundedness, relevance, and precision/recall
• Bias and fairness checks, safety/guardrail, and PII/data‑leakage validation

Languages & Frameworks

Python • Pytest

• Proficient in Python for automation
• Hands‑on with Pytest, or equivalent
• Build, scale, and maintain automation frameworks end‑to‑end

Secondary Skill

UI & API Testing

UI Automation • REST/GraphQL APIs • Contract Testing

• End‑to‑end browser testing across modern web applications. Well versed with BDD frameworks like Cucumber/SpecFlow
• Validates REST/GraphQL APIs across microservices for correctness and edge cases
• Bridges UI and API layers for integrated test coverage
• Experience with tools like Postman, Insomnia, Newman

Cloud, Infrastructure & CI/CD

AWS • SQL • CI/CD Pipelines

• Working knowledge of AWS/Azure services for test environments
• Uses SQL to query, validate, and debug data confidently
• Integrates test suites into Jenkins, GitHub Actions, GitLab CI, etc.