Senior Software QA Engineer, AI Software/Platform Team

Jobtailor; Milpitas (CA); On-site; USD 120,000 - 160,000; Full time

Job summary

Jobtailor is seeking a Lead QA Architect to own quality strategy for large-scale Generative AI products. You will design validation approaches for AI services, distributed systems, and customer experiences, while shaping scalable testing across pipelines, APIs, UIs, and multi-tenant SaaS environments.

You will mentor engineers, establish release-readiness gates, and drive data-driven decision making to improve AI platform reliability and resilience across cross-functional teams.

Qualifications

• Design and implement test strategies and automation for distributed software systems and cloud platforms.
• Validate AI services, frontend and backend systems with REST/GraphQL APIs and CI/CD pipelines.
• Evaluate model quality and degradation in Generative AI workloads.
• Define release-quality criteria and data-driven Go/No-Go decisions.
• Mentor QA engineers and drive quality-first engineering practices across teams.

Responsibilities

• Lead the quality strategy for large-scale Generative AI products and platforms.
• Define validation approaches forAI services, distributed systems, and end-to-end customer experiences.
• Design scalable testing and automation across AI pipelines, APIs, UIs, and multi-tenant SaaS architectures.
• Establish release-readiness practices using quality gates and data-driven criteria.
• Mentor QA engineers and advance quality-first practices across teams.

Skills

Quality Strategy Leadership

Test Strategy Design

Python Development

Automated Testing Frameworks

Generative AI Testing

Education

Bachelor’s Degree in Engineering or related field

Master’s Degree in Computer Science or Engineering

PhD with 2 years experience

Tools

Pytest

Playwright

Selenium

Cursor

Claude

Codex CLI

Kubernetes

ArgoCD

Job description

• Lead the quality strategy for large-scale Generative AI products and platforms
• Define validation approaches for AI services, distributed systems, and end-to-end customer experiences
• Design scalable testing and automation across AI pipelines, APIs, user interfaces, federated microservices, and multi-tenant SaaS architectures
• Establish release-readiness practices using quality gates, test traceability, production-representative environments, and data-driven Go/No-Go criteria
• Analyze model degradation, system failures, and test instability
• Drive root-cause analysis to improve AI platform reliability and resilience
• Partner with Engineering, Product, UX, DevOps, Documentation, and Customer Engineering
• Mentor QA engineers and advance quality-first engineering practices across teams

Requirements

• Bachelor’s Degree in Engineering or a related field and 6 years of relevant experience, or Master’s Degree in Computer Science or Engineering and 4 years of experience, or PhD and 2 years of practical experience
• Experience designing and implementing test strategies, automation frameworks, and test infrastructure for distributed software systems, cloud platforms, or SaaS products
• Python development experience
• Experience with automated testing frameworks such as Pytest, Playwright, Selenium, or equivalent
• Experience validating frontend and backend systems, including REST and/or GraphQL APIs, service integrations, CI/CD pipelines, and distributed microservice architectures
• Experience defining release-quality criteria and diagnosing system failures using test results, logs, metrics, and other engineering data
• Experience testing Generative AI, AI/ML platforms, AI agents, LLM-powered applications, or other AI-based products
• Experience evaluating model quality and degradation
• Experience designing AI-agent-assisted QA workflows using tools such as Cursor, Claude, Codex CLI, or equivalent
• Experience with agent-based or MCP-based test orchestration
• Experience testing multi-tenant cloud platforms and managing AI test environments involving datasets, model versions, feature flags, Kubernetes, and ArgoCD or similar deployment technologies
• Experience building scalable test frameworks and establishing practices across multiple engineering teams
• Knowledge of security, privacy, data governance, or compliance requirements relevant to enterprise SaaS and AI products

Core Competencies

Demonstrates expertise in leading quality strategies for Generative AI products, including designing scalable testing and automation frameworks, validating AI services, and establishing release-readiness practices. Proficient in Python development and automated testing frameworks, with a strong focus on improving AI platform reliability and resilience.

Highest-signal resume keywords

• Quality Strategy Leadership
• Test Strategy Design
• Python Development
• Automated Testing Frameworks
• Generative AI Testing

Hard Skills

• Test Strategy Design
• Automation Frameworks
• Python Development
• REST APIs
• GraphQL APIs
• CI/CD Pipelines
• Distributed Systems Testing
• Model Quality Evaluation
• Test Orchestration
• Multi-Tenant Cloud Platforms

Soft Skills

• Mentoring
• Collaboration
• Root-Cause Analysis

Industry Keywords

• Generative AI
• AI/ML Platforms
• Data Governance
• Compliance Requirements
• Quality Gates

Tools & Technologies

• Pytest
• Playwright
• Selenium
• Cursor
• Claude
• Codex CLI
• Kubernetes
• ArgoCD