Quality Engineer for AI/LLM Testing in Legal Tech

Zipliens; Franklin (TN); On-site; USD 77,000 - 104,000; Full time

Benefits offered by this job

Private Health Care Plan (Medical, ???

Company HSA contributions

Flexible Spending Accounts

Company-Paid Short-Term Disability

Voluntary Long-Term Disability, Life,

401(k) Plan with Company Match

Paid Time Off

Parental Leave

Job summary

Zipliens is seeking a Quality Engineer to own quality for a product area within our lien-resolution software stack used by personal injury law firms. You will shape test strategies, build automated suites, and ensure reliable releases in a data-heavy environment that uses AI/LLMs for structure extraction from medical records and lien documents.

You will collaborate with engineers to refine Acceptance Criteria, balance risk, and triage defects end-to-end, with a focus on robust automation and

Qualifications

• 3+ years on a QA or software development team, including owning test coverage for a feature or product area.
• Strong skills in at least one of exploratory testing and test design, test automation and infrastructure, or AI/LLM evaluation, with working proficiency in a second.
• Hands-on experience building and maintaining automated test suites in a modern framework (Playwright, Cypress, pytest, or similar), including diagnosing flaky tests and fixing their root causes.
• Comfortable reading and writing code. Our backend is Python/Django and our frontend is React, and you should be able to read a change in either and understand what it affects and what it puts at risk.
• Experience integrating tests into CI pipelines and deciding which checks should gate a merge or a deploy.
• Strong SQL. Much of what we test lives in PostgreSQL, and verifying a workflow often means verifying the data it produced as well as what the UI shows.
• Hands-on experience testing LLM or ML features, whether in production or through prototypes and experimentation: building golden datasets, measuring output quality, and understanding how these systems fail.
• Experience using or experimenting with agentic coding tools, and good judgment about when their output needs a closer look.
• Clear written communication. You'll be writing defect reports, test plans, and release sign-offs for engineers and product, and explaining risk to non-technical stakeholders.

Responsibilities

• Own quality for a feature or product area, including the risk-based test strategy, the regression scope, and the release sign-off for that area.
• Build and maintain the automated test suites for your area and keep them integrated with CI. When tests fail or become flaky, you'll troubleshoot them and fix the underlying cause.
• Run exploratory testing against the lien workflow and develop a detailed understanding of the applications your team owns, including their risk areas and failure modes.
• Test our LLM-based features. This includes building the evaluation suite for a feature, setting acceptance thresholds with product (for example, precision and recall targets for extraction), checking LLM-judge agreement against human labels, and adding adversarial and prompt-injection cases.
• Use agentic coding tools to build and maintain automation, and review AI-generated tests before they merge.
• Join refinement early enough to shape Acceptance Criteria with the Product Owner, flag testability gaps before development starts, and recommend coverage and risk priorities for your area.
• Shepherd defects for your area from triage through resolution, writing clear, reproducible reports that engineers can act on.
• Work directly with the engineers on your team day to day. QA at Zipliens is embedded in the engineering teams rather than run as a separate function.

Skills

QA experience

Test automation

CI pipelines

SQL

Python

AI/LLM evaluation

Code reading

Communication

Tools

Playwright

Cypress

pytest

Job description

Zipliens is seeking a Quality Engineer to own quality for a product area within our lien-resolution software stack used by personal injury law firms. You will shape test strategies, build automated suites, and ensure reliable releases in a data-heavy environment that uses AI/LLMs for structure extraction from medical records and lien documents.

You will collaborate with engineers to refine Acceptance Criteria, balance risk, and triage defects end-to-end, with a focus on robust automation and