Senior Quality Engineer: AI/LLM Testing & Automation Lead

Zipliens; Franklin (TN); On-site; USD 98,000 - 132,000; Full time

Benefits offered by this job

Private Health Care Plan

HSA contributions for HDHP

Flexible Spending Accounts (Health & D

Company-Paid Short-Term Disability

Voluntary Long-Term Disability, Life,

401(k) Plan with Company Match

Paid Time Off (Vacation, Sick Time & 旄

Parental Leave

Job summary

Zipliens, a leading lien resolution company in the United States, seeks a Senior Quality Engineer to set quality standards for a growing engineering team. You will drive testing strategy, automate and gate CI/CD, and oversee AI/LLM evaluation and risk-based testing to ensure data-heavy products perform reliably at scale.

You will mentor engineers, shape acceptance criteria with product, and evaluate new tools and capabilities while collaborating across teams to define release criteria and

Qualifications

• 5+ years on a QA or software development team, with experience across a range of products and test approaches.
• Expert-level skills in at least one of exploratory testing and test design, test automation and infrastructure, or AI/LLM evaluation, and solid proficiency in the other two.
• Experience setting test standards and choosing frameworks for a team, and getting the team to adopt them.
• Significant experience building and owning automation in a modern framework (Playwright, Cypress, pytest, or similar), including designing the CI/CD gating strategy around it.

Responsibilities

• Lead quality for medium to large features, including ones that cross team boundaries; own risk analysis, test strategy, release criteria, and final sign-off.
• Set and advocate for the team's test standards and frameworks; define sufficient coverage and passing build criteria.
• Own the automation and CI/CD gating strategy: which suites run at which stage, gates for merge vs deploy, and reducing flakiness.
• Own the evaluation methodology for our LLM features, including sampling, rubric design, and monitoring post-release quality.
• Set standards for validating AI-generated output in product, tests, and tooling.
• Collaborate with Product Owner to define Acceptance Criteria and release criteria; act as primary QA point for cross-functional planning.
• Evaluate and introduce new tools and AI capabilities for the team.
• Mentor quality engineers and coach on exploratory technique and risk-based testing; help scale the team.

Skills

Exploratory testing

Test automation

AI/LLM evaluation

Mentoring

Tools

Playwright

Cypress

pytest

Python

Job description

Zipliens, a leading lien resolution company in the United States, seeks a Senior Quality Engineer to set quality standards for a growing engineering team. You will drive testing strategy, automate and gate CI/CD, and oversee AI/LLM evaluation and risk-based testing to ensure data-heavy products perform reliably at scale.

You will mentor engineers, shape acceptance criteria with product, and evaluate new tools and capabilities while collaborating across teams to define release criteria and