Senior AI-First Web QA Architect

Adobe Inc.; California (MO); On-site; USD 113,000 - 222,000; Full time

Job summary

Adobe seeks a Senior Software Quality Engineer to lead quality strategy for a new AI-first web application. You will own end-to-end testing across prompt, agent, and in-browser experiences, building fast, deterministic automation and evaluation harnesses.

You will drive test strategy for AI/ML features, emphasize performance, accessibility, and cross-device coverage, and embed quality into CI/CD with actionable telemetry and gates.

Qualifications

• 8+ years of experience in customer-facing software quality / SDET roles.
• Proficient in Playwright or Cypress with TypeScript/JavaScript; Python a plus.
• Experience designing test strategies for AI/ML features - prompt regression, evaluation datasets, model drift, safety testing, or equivalent.
• Strong API, performance and load testing, and observability tooling fundamentals.

Responsibilities

• Own the end-to-end quality strategy for a new AI-first web application, spanning the prompt and agent layer, the in-browser experience, and the generated output.
• Compose, build, and maintain modern web test automation (Playwright / Cypress, TypeScript/JavaScript) and underlying infrastructure for fast, deterministic testing.
• Stand up evaluation harnesses for automated workflows: golden prompt suites, regression baselines, model/version comparisons, hallucination and safety checks, and human-in-the-loop review pipelines.
• Ensure comprehensive quality for a real-time web product - performance, resilience, accessibility (WCAG), and cross-browser/cross-device coverage.
• Embed quality into CI/CD: build release gates that catch real regressions, reduce flake, and turn production telemetry into pre-submit signal.
• Work closely with engineering, product, and design to build testability into the process early. Guide engineers on test strategy and automation patterns.

Skills

SDET experience

Automation testing

JavaScript/TypeScript

API testing

Education

Bachelor's degree in Computer Science or related field

Tools

Playwright

Cypress

Python

OpenTelemetry

Datadog

Splunk

Job description

Adobe seeks a Senior Software Quality Engineer to lead quality strategy for a new AI-first web application. You will own end-to-end testing across prompt, agent, and in-browser experiences, building fast, deterministic automation and evaluation harnesses.

You will drive test strategy for AI/ML features, emphasize performance, accessibility, and cross-device coverage, and embed quality into CI/CD with actionable telemetry and gates.