Lead Software QA Engineer, AI Software/Platform Team

Jobtailor; Milpitas (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Jobtailor is seeking a senior QA leader to own the quality strategy for large-scale Generative AI products. You will define validation approaches for AI services, distributed systems, and end-to-end customer experiences.

You will design scalable testing across AI pipelines, APIs, UIs, and multi-tenant SaaS, and establish release-readiness using quality gates and data-driven Go/No-Go criteria. Mentorship and cross-team collaboration are essential.

Qualifications

• Design and implement test strategies, automation frameworks, and test infrastructure for distributed software systems, cloud platforms, or SaaS products.
• Strong Python development experience and familiarity with automated testing tools such as Pytest, Playwright, or Selenium.
• Experience validating REST and GraphQL APIs, CI/CD pipelines, and distributed microservice architectures.
• Ability to define release-quality criteria and conduct root-cause analysis from logs and metrics.
• Preferred: experience testing Generative AI, AI/ML platforms, or AI agents; experience AI-agent-assisted QA workflows.

Responsibilities

• Lead the quality strategy for large-scale Generative AI products and platforms.
• Define validation approaches for AI services, distributed systems, and end-to-end customer experiences.
• Design scalable testing and automation across AI pipelines, APIs, UI, microservices, and multi-tenant SaaS.
• Establish release-readiness practices using quality gates and data-driven Go/No-Go criteria.
• Analyze model degradation, system failures, and test instability; drive root-cause analysis.
• Partner with Engineering, Product, UX, DevOps, Documentation, and Customer Engineering.
• Mentor QA engineers and advance quality-first practices across teams.

Skills

Python Development

Automated Testing Frameworks

Test Strategy Design

Distributed Systems Testing

Root-Cause Analysis

Collaboration

Leadership

Quality Gates Definition

Data-Driven Go/No-Go Criteria

Education

BS in Computer Science or Engineering with 12+ years

MS in Computer Science or related field with 8+ years

PhD with 5+ years

Tools

Pytest

Playwright

Selenium

Kubernetes

ArgoCD

Cursor

Claude

Codex CLI

Job description

• Lead the quality strategy for large-scale Generative AI products and platforms
• Define validation approaches for AI services, distributed systems, and end-to-end customer experiences
• Design scalable testing and automation across AI pipelines, APIs, user interfaces, federated microservices, and multi-tenant SaaS architectures
• Establish release-readiness practices using quality gates, test traceability, representative environments, and data-driven Go/No-Go criteria
• Analyze model degradation, system failures, and test instability
• Drive root-cause analysis and improve AI platform reliability and resilience
• Partner with Engineering, Product, UX, DevOps, Documentation, and Customer Engineering
• Mentor QA engineers and advance quality-first engineering practices across teams

Requirements

• BS in Computer Science, Engineering, or a related field with 12+ years of relevant experience; or a Master’s of Science Degree with 8+ years of related experience, or a PhD with 5+ years of related experience
• Experience designing and implementing test strategies, automation frameworks, and test infrastructure for distributed software systems, cloud platforms, or SaaS products
• Python development experience
• Experience with automated testing frameworks such as Pytest, Playwright, Selenium, or equivalent
• Experience validating frontend and backend systems, REST and/or GraphQL APIs, service integrations, CI/CD pipelines, and distributed microservice architectures
• Experience defining release-quality criteria
• Experience diagnosing complex system failures using test results, logs, metrics, and other engineering data
• Preferred: experience testing Generative AI, AI/ML platforms, AI agents, LLM-powered applications, or other AI-based products
• Preferred: experience designing AI-agent-assisted QA workflows using Cursor, Claude, Codex CLI, or equivalent
• Preferred: experience testing multi-tenant cloud platforms and AI test environments involving datasets, model versions, feature flags, Kubernetes, and ArgoCD
• Preferred: experience building scalable test frameworks and establishing quality practices across multiple engineering teams
• Preferred: knowledge of security, privacy, data governance, or compliance requirements for enterprise SaaS and AI products

Core Competencies

Demonstrates expertise in leading quality strategies for Generative AI products, including designing scalable testing and automation frameworks for distributed systems and cloud platforms. Proficient in defining release-quality criteria and conducting root-cause analysis to enhance platform reliability.

Highest-signal resume keywords

• Generative AI Testing
• Python Development
• Automated Testing Frameworks
• Test Strategy Design
• Cloud Platform Validation

Hard Skills

• Test Strategy Design
• Automation Frameworks
• Distributed Systems Testing
• REST APIs Validation
• GraphQL APIs Validation
• CI/CD Pipeline Testing
• Model Degradation Analysis
• Root-Cause Analysis
• Quality Gates Definition
• Data-Driven Go/No-Go Criteria

Soft Skills

• Mentoring
• Collaboration
• Leadership

Industry Keywords

• Generative AI
• AI/ML Platforms
• Multi-Tenant SaaS
• Data Governance
• Compliance Requirements

Tools & Technologies

• Pytest
• Playwright
• Selenium
• Kubernetes
• ArgoCD