Senior AI/Agentic QA Engineer (SDET) - Remote

Qualia; Oregon; Hybrid; USD 140,000 - 200,000; Full time

Benefits offered by this job

Health plans

401k

Commuter benefits

Parental leave

Flexible time off

Onboarding program

All hands meetings

Job summary

Qualia is hiring a Senior AI/Agentic Quality Infrastructure SDET to build production-grade AI-assisted test infrastructure and agentic validation systems. You’ll design and implement AI-enabled evaluation harnesses, observability platforms, and tooling to validate complex workflows across apps, databases, and CI/CD pipelines.

You will mentor engineers, evolve testing architectures, and collaborate with SRE, Platform, and Engineering teams to deliver scalable, reliable testing solutions in a

Qualifications

• 5+ years as an SDET or software engineer building test infrastructure, developer tooling, or observability systems.
• Strong programming in JavaScript/TypeScript, including backend services and infrastructure.
• Solid fundamentals in system design, distributed systems, APIs, and production ops.
• Experience with CI/CD, release orchestration, test infrastructure, and observability systems.
• Hands-on with LLMs, AI-assisted engineering, or agentic systems including tool calling and multi-step workflows.
• Familiarity with LLM/agent evaluation and probabilistic/non-deterministic testing challenges.
• Understanding of testing pyramid and where validation belongs (unit to agentic layers).
• Mentoring engineers and raising quality bar across teams.

Responsibilities

• Build and maintain test infrastructure, AI/ML evaluation systems, and agentic validation harnesses as production-grade software.
• Design agentic test harnesses enabling AI agents to interact with apps, browsers, APIs, databases, logs, and CI/CD.
• Develop AI/ML evaluation frameworks with deterministic and model-based evaluators and quality signals.
• Incorporate observability/telemetry into automation and agentic infra with logs, metrics, and traces.
• Differentiate product/test/infrastructure/AI failures to speed root-cause analysis.
• Own test automation frameworks and migration paths (e.g., Cypress to Playwright) and scale CI/CD provisioning.
• Apply AI to engineering-quality problems and build guardrails for evidence-backed conclusions.
• Collaborate with software, quality, SRE, and platform teams to build reusable platforms and mentor others.

Skills

SDET experience

JavaScript/TypeScript

System design

CI/CD

LLM/agentic systems

Mentoring

Tools

Playwright

Jest

Postman

Locust

Gatling

JMeter

k6

Jenkins

GitHub Actions

ArgoCD

Spinnaker

Harness

Datadog

OpenTelemetry

MongoDB

Kubernetes

AWS

Google Cloud

Job description

Qualia is hiring a Senior AI/Agentic Quality Infrastructure SDET to build production-grade AI-assisted test infrastructure and agentic validation systems. You’ll design and implement AI-enabled evaluation harnesses, observability platforms, and tooling to validate complex workflows across apps, databases, and CI/CD pipelines.

You will mentor engineers, evolve testing architectures, and collaborate with SRE, Platform, and Engineering teams to deliver scalable, reliable testing solutions in a