AI SDET: Architect Self-Healing Testing & GenAI Pipelines

TalentOla; Sunnyvale (CA); On-site; USD 150,000 - 190,000; Full time

Job summary

TalentOla is seeking an innovative AI Software Development Engineer in Test (AI-SDET) to join the client’s quality engineering team, bridging QA and AI by designing self-healing test automation frameworks, validating LLMs, and integrating AI agents into CI/CD pipelines.

The ideal candidate has 6 to 8 years of hands-on experience in software quality engineering, test automation and AI/ML development, with a Bachelor’s degree in computer science or related field and proficiency in

Qualifications

• Bachelor’s degree in computer science or related field.
• Proficiency in Python/Java/Scala.
• Proficiency with Playwright and modern tools.
• Experience with LLM APIs and agent frameworks like LangChain.
• Cloud AI platforms with Azure OpenAI / AWS OpenAI.
• Strong CI/CD and Docker knowledge.
• Experience validating RESTful APIs, microservices, databases (SQL/NoSQL).
• Retail domain experience preferred but not required.

Responsibilities

• AI Framework Architecture: Architect and maintain enterprise-grade, scalable automation frameworks.
• GenAI and Agentic automation: Build intelligent test agents, automated prompt-engineering validators, and self-healing test execution scripts.
• Model and Evals validation: Design and maintain evaluation frameworks covering hallucination detection, roundedness, toxicity, bias and safety guardrails.
• Synthetic Data Generation: Design AI-driven data generation pipelines to simulate complex, production-like testing scenarios.
• ML Pipeline testing: Validate RAG architectures, embedding quality, retrieval accuracy, and vector search relevance.
• Intelligent testing: Build smart test selection and impact analysis systems that predict which test suits to run based on specific code changes.
• Framework Development: Develop and enhance scalable, robust automation scripts for UI, API, and backend validation using modern languages and frameworks.
• CI CD Integration: Work alongside MLOps teams to build continuous testing and automated verification pipelines.

Skills

Python/Java/Scala

Playwright

LLM APIs

LangChain

Azure OpenAI

AWS OpenAI

CI/CD

Docker

RESTful APIs

SQL/NoSQL

Retail Domain

Education

Bachelor’s degree in computer science

Tools

Docker

Job description

TalentOla is seeking an innovative AI Software Development Engineer in Test (AI-SDET) to join the client’s quality engineering team, bridging QA and AI by designing self-healing test automation frameworks, validating LLMs, and integrating AI agents into CI/CD pipelines.

The ideal candidate has 6 to 8 years of hands-on experience in software quality engineering, test automation and AI/ML development, with a Bachelor’s degree in computer science or related field and proficiency in