Senior AI Quality Engineer — Remote, AI Testing & Strategy

Jobicy; United States; Remote; USD 136,000 - 187,000; Full time

Benefits offered by this job

Health benefits

Remote-first

Work from home stipend

Job summary

StackAdapt is seeking a Staff Quality Engineer to lead the Automation and Intelligence Pillar, owning the end-to-end quality strategy for AI-driven features including LLM-powered workflows. You will design evaluation frameworks, build scalable test harnesses, and mentor a growing QE organization while accelerating releases.

This role requires deep expertise in API/UI test automation, CI/CD, and experience with AI testing, with collaboration across ML/DS, product, and engineering to raise the bar

Qualifications

• 7+ years of professional QA and test automation experience.
• Experience with API and UI test automation frameworks.
• Fluency in at least one of TypeScript, JavaScript, Python, or Java.
• Experience testing AI/agentic systems and using AI to scale automation.
• CI/CD and web API/backend testing expertise.
• Knowledge of React and data-testids.
• Ability to mentor cross-functional teams on tooling and testing plans.

Responsibilities

• Own end-to-end quality strategy for the Automation and Intelligence Pillar pod’s AI such as Ivy, from test architecture through release readiness.
• Design evaluation frameworks for non-deterministic systems, including LLM output quality, agent behaviour, regression detection, and guardrail testing.
• Develop reliable, scalable automated test suites and tooling addressing functional, system, API, integration, UI, and regression testing, integrated into CI/CD.
• Establish quality metrics and benchmarks (accuracy, latency, cost, hallucination/failure rates) to guide shipping decisions.
• Partner with Engineering Managers, Product Managers, Designers, ML/DS, and other teams to align requirements and definitions of done.
• Mentor QE org, promote QE best practices, and uplift testing for AI systems company-wide.
• Participate in planning, code reviews, and architectural decisions, advocating for testability and reliability.
• Provide timely test results to stakeholders in the pod and QA teams.

Skills

QA strategy

Test automation

API testing

UI testing

Playwright

Cypress

TypeScript

Python

CI/CD

React testing

Tools

JUnit

TestRail

Jenkins

GitHub Actions

CircleCI

Playwright

Cypress

Job description

StackAdapt is seeking a Staff Quality Engineer to lead the Automation and Intelligence Pillar, owning the end-to-end quality strategy for AI-driven features including LLM-powered workflows. You will design evaluation frameworks, build scalable test harnesses, and mentor a growing QE organization while accelerating releases.

This role requires deep expertise in API/UI test automation, CI/CD, and experience with AI testing, with collaboration across ML/DS, product, and engineering to raise the bar