AI First – QA Engineer – USA (Remote)

Digitalt3; Northern (KY); Remote; USD 140,000 - 190,000; Full time

Job summary

DigitalT3 is seeking a Principal SDET to anchor quality engineering for an AI‑first organization. You will define the long‑term quality architecture and set org‑wide testing standards to ensure GenAI components meet reliability, correctness, and performance standards.

You will collaborate with Tech Leads and product leadership to raise the quality bar, mentor teams, and champion AI‑powered testing tools and observability across the SDLC.

Qualifications

• 8–10 years in SDET, quality engineering, or test architecture roles.
• Expert knowledge of CI/CD pipelines and cloud test infra (AWS preferred).
• Experience leading quality initiatives across cross‑functional teams and influencing engineering standards.
• Strong communication and stakeholder management to present quality metrics to leadership.
• Proven ability to mentor senior engineers and grow the capability of a quality engineering team.

Responsibilities

• Define and own the overall quality engineering strategy and test architecture across all product lines and teams.
• Establish org‑wide standards, frameworks, and best practices for test automation, performance testing, and reliability engineering.
• Lead the design of scalable, cloud‑native test infrastructure on AWS, including containerized environments and distributed test execution.
• Drive quality governance: KPIs, coverage targets, and quality gates across the SDLC.
• Partner with Tech Leads to ensure testability in system and API designs from the start.
• Lead Agile quality practices; own sprint‑level and release‑level quality reporting to leadership.
• Mentor senior and mid‑level SDETs; develop engineering capability across the quality organization.
• Evaluate, select, and champion testing tools and platforms, including AI‑powered quality solutions.

Skills

SDET

Quality engineering

Test architecture

CI/CD

AWS

Test automation

Performance testing

AI testing

Mentoring

Stakeholder management

Leadership

Tools

Jenkins

Docker

Kubernetes

Selenium

Test harnesses

Job description

Location: 100% Remote – Anywhere in the U.S.

Experience: 8–10 years of experience in SDET, quality engineering, or test architecture roles.

Role Overview

We’re looking for a Principal SDET to serve as the quality engineering anchor for our AI-first engineering organization. In this role, you’ll define the long‑term quality architecture, establish org‑wide testing standards, and drive the strategy for how AI transforms quality engineering at scale. You’ll work closely with Tech Leads, Principal Engineers, and product leadership to ensure our systems — including their GenAI components — meet the highest standards of reliability, correctness, and performance.

Key Responsibilities

Quality & Test Engineering

• Define and own the overall quality engineering strategy and test architecture across all product lines and engineering teams.
• Establish org‑wide standards, frameworks, and best practices for test automation, performance testing, and reliability engineering.
• Lead the design of scalable, cloud‑native test infrastructure on AWS, including containerized test environments and distributed test execution.
• Drive quality governance: define KPIs, coverage targets, and quality gates across the SDLC.
• Partner with Tech Leads and Principal Engineers to ensure testability is built into system and API designs from the start.
• Lead the adoption of Agile quality practices; own sprint‑level and release‑level quality reporting to leadership.
• Mentor senior and mid‑level SDETs; develop engineering capability across the quality organization.
• Evaluate, select, and champion testing tools and platforms — including AI‑powered quality solutions.

AI-Augmented Testing

• Own the strategy for testing GenAI and LLM‑powered systems at scale — including output validation, semantic similarity testing, hallucination detection, and behavioral regression.
• Architect evaluation frameworks for LLM applications, defining metrics, benchmarks, and automated quality gates for AI features.
• Partner with AI engineering teams to define testability requirements for LLM pipelines, RAG systems, and prompt frameworks.
• Build and evolve monitoring and observability approaches for AI system quality in production.
• Drive organization‑wide adoption of GenAI tools to improve test productivity, coverage, and insight generation.
• Represent quality engineering in AI architecture discussions, ensuring new AI systems are built with measurability and testability in mind.

Candidate Requirements

• 8–10 years of experience in SDET, quality engineering, or test architecture roles.
• Deep expertise across the full spectrum of testing: functional, integration, API, performance, security, and reliability.
• Proven track record of defining and owning test strategy and quality architecture across large engineering organizations.
• Expert knowledge of CI/CD pipelines, cloud test infrastructure (AWS preferred), and containerized environments.
• Experience leading quality initiatives across cross‑functional teams and influencing engineering standards.
• Strong communication and stakeholder management skills; able to present quality metrics and strategy to leadership.
• Proven ability to mentor senior engineers and grow the capability of a quality engineering team.

Good to Have

• Experience designing evaluation frameworks for LLM or GenAI systems.
• Background in MLOps, AI observability, or model quality monitoring.
• Familiarity with chaos engineering and site reliability engineering (SRE) practices.
• Experience with open‑source contributions to testing frameworks or AI quality tooling.
• Background in developer experience (DX) improvements that increase testability and code quality org‑wide.

As set forth in DigitalT3’s Equal Employment Opportunity policy, we do not discriminate on the basis of any protected group status under any applicable law.