Senior AI-First Web Quality Engineer

Embedded Shishya; United States; Remote; USD 113,000 - 222,000; Full time

Job summary

Adobe is seeking a Senior Software Quality Engineer to define and drive the quality strategy for a new AI-first web application. You will build fast, reliable test systems and shape the automation landscape across the prompt/agent layer, in-browser experience, and generated output.

You will implement modern web test automation using TypeScript/JavaScript, with Python as a plus, and establish evaluation harnesses for regression, model comparisons, and safety checks within a fast-paced

Qualifications

• Bachelor's degree or equivalent experience in Computer Science or a related field.
• 8+ years of experience in customer-facing software quality / SDET positions.
• Automation experience with modern web frameworks (Playwright, Cypress) and strong coding in TypeScript/JavaScript; Python is a plus.
• Experience designing test strategy for AI / ML / LLM-powered features —prompt regression, evaluation datasets, model drift, safety testing, or equivalent.
• Strong fundamentals in API testing, performance and load testing, and observability tooling (Datadog, Splunk, Sentry, OpenTelemetry, or similar).
• Excellent collaboration and communication; able to influence engineers and PMs early in the build cycle.

Responsibilities

• Own the end-to-end quality strategy for a new AI-first web application, spanning the prompt and agent layer, the in-browser experience, and the generated output.
• Compose, build, and maintain modern web test automation (Playwright / Cypress, TypeScript/JavaScript) and the underlying infrastructure that keeps it fast, deterministic, and developer-friendly.
• Stand up evaluation harnesses for automated workflows: golden prompt suites, regression baselines, model/version comparisons, hallucination and safety checks, and human-in-the-loop review pipelines.
• Ensure comprehensive quality for a real-time web product – performance, resilience, accessibility (WCAG), and cross-browser/cross-device coverage.
• Embed quality into CI/CD: build release gates that catch real regressions, reduce flake, and turn production telemetry into pre-submit signal.
• Work closely with engineering, product, and design to build testability into the process early. Guide engineers on test strategy, automation patterns, and adopting a shift-left outlook.

Skills

Automation

TypeScript/JavaScript

Playwright/Cypress

CI/CD

SDET

Education

Bachelor's degree in CS or related field

Tools

Playwright

Cypress

Python

Job description

Adobe is seeking a Senior Software Quality Engineer to define and drive the quality strategy for a new AI-first web application. You will build fast, reliable test systems and shape the automation landscape across the prompt/agent layer, in-browser experience, and generated output.

You will implement modern web test automation using TypeScript/JavaScript, with Python as a plus, and establish evaluation harnesses for regression, model comparisons, and safety checks within a fast-paced