Senior AI QA Engineer — Full-Stack & LLM Quality

XY.ai Inc.; United States; Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

Equity

Job summary

XY.AI Labs is building an AI-powered, fully-automated quality and release platform in a health-tech startup. The role emphasizes hands-on engineering across Python backend and Node/Next.js frontend, with GitHub Actions pipelines and containerized environments.

You will design prompt-stress suites, living dashboards, and governance for progressive releases while mentoring teammates and shaping our QA strategy as we scale from day 0 to unicorn.

Qualifications

• Proven design of end-to-end automation frameworks and zero-downtime releases.
• Experience mentoring dev teams in quality practices.
• Hands-on testing of ML/LLM features, prompt engineering, and bias checks.

Responsibilities

• Build and maintain test infrastructure in Python (backend) and Node/Next.js (frontend).
• Architect GitHub Actions pipelines for unit, API, UI, and performance tests.
• Implement container-first test environments (Docker/Kubernetes) and IaC for staging and releases.
• Create dashboards showing coverage, MTTR, and AI performance drift for leadership.

Skills

QA automation

Full-stack engineering

LLM evaluation

Prompt engineering

Mentoring

Tools

Python

Node.js

Next.js

Pydantic

GitHub Actions

Selenium

Playwright

Docker

Kubernetes

Job description

XY.AI Labs is building an AI-powered, fully-automated quality and release platform in a health-tech startup. The role emphasizes hands-on engineering across Python backend and Node/Next.js frontend, with GitHub Actions pipelines and containerized environments.

You will design prompt-stress suites, living dashboards, and governance for progressive releases while mentoring teammates and shaping our QA strategy as we scale from day 0 to unicorn.