Senior QA Engineer

S27a; Northern, New York (KY, NY); Hybrid; USD 100,000 - 140,000; Full time

Job summary

Dili is seeking a Senior QA Engineer to own automated product quality for its compliance engine, focusing on formal validation and regression prevention. You will translate regulations into executable tests and work with engineers and Compliance.

You will build a registry of compliance scenarios, automate tests, and expand coverage across federal, state, and program requirements, enabling CI-ready quality gates and release readiness.

Qualifications

• Strong experience in software QA, quality engineering, or test automation.
• Strong Python skills.
• Experience building automated integration and end-to-end test suites.
• Experience integrating testing into CI/CD and release processes.
• Ability to test a Python/React application in an AWS environment.
• Ability to systematically reason about complex combinations of rules, inputs, and expected outcomes.
• Ability to turn written requirements into precise, deterministic test cases.
• Strong attention to traceability: not just “did this test pass?”, but “why is this the expected behavior, and what requirement supports it?”
• Ability to work closely with engineers and domain experts to resolve ambiguity.

Responsibilities

• Build and maintain a registry of compliance scenarios across supported frameworks.
• Translate regulations, agency guidance, and internal compliance interpretations into structured, testable cases.
• Include citations to the authoritative regulatory or program sources behind expected behavior.
• Develop automated tests that formally verify our compliance engine.
• Test interactions between overlapping federal, state, and program-specific requirements.
• Cover edge cases, exceptions, remediation flows, and previously identified production issues.
• Make scenarios easy for our Compliance team to review, update, and extend.
• Expand automated coverage across high-risk product workflows.
• Build and improve integration, API, end-to-end, and regression test suites.
• Establish automated quality gates and release-readiness criteria.
• Improve visibility into scenario coverage, failures, and known gaps.

Skills

QA engineering

Python

Test automation

CI/CD integration

End-to-end testing

Regression testing

Tools

Playwright

QA Wolf

CI/CD tools

Job description

Senior QA Engineer

About Dili

Dili builds software that helps construction and clean-energy companies manage complex prevailing wage, apprenticeship, and labor compliance requirements.

We started Dili because labor compliance is still handled through a mix of spreadsheets, PDFs, manual review, and institutional knowledge, despite determining whether some of the largest construction projects in the country are compliant.

Today, we’re a team of roughly 30, including a small engineering team working closely with our in-house Compliance team. We’re building the infrastructure to turn complex regulatory requirements into software that can be evaluated consistently, explained clearly, and audited.

About the role

Our compliance engine evaluates payroll, timekeeping, worker classifications, prevailing wage rates, apprenticeship requirements, and documentation across frameworks including federal Davis-Bacon, IRA requirements, state prevailing wage laws, and programs such as NYSERDA and Illinois Shines.

We’re looking for a Senior QA Engineer to own automated product quality, with a particular focus on formally validating our compliance engine and preventing regressions.

This is not primarily a manual QA role. You’ll build the systems that let us enumerate the compliance scenarios we support, tie them to the underlying regulations, automate their validation, and confidently determine whether a release is correct.

You’ll work directly with engineers and our Compliance team to turn regulations and interpretations into deterministic, executable tests.

What you'll own

• Build and maintain a registry of compliance scenarios across supported frameworks

• Translate regulations, agency guidance, and internal compliance interpretations into structured, testable cases

• Include citations to the authoritative regulatory or program sources behind expected behavior

• Develop automated tests that formally verify our compliance engine

• Test interactions between overlapping federal, state, and program-specific requirements

• Cover edge cases, exceptions, remediation flows, and previously identified production issues

• Make scenarios easy for our Compliance team to review, update, and extend

• Expand automated coverage across high-risk product workflows

• Build and improve integration, API, end-to-end, and regression test suites

• Establish automated quality gates and release-readiness criteria

• Improve visibility into scenario coverage, failures, and known gaps

What we're trying to build

We want our test infrastructure to become an executable specification of the Dili compliance engine.

For each scenario, we should be able to define:

• the applicable regulation or program requirement, with citation

• the relevant project, payroll, worker, wage, and apprenticeship inputs

• the expected compliance determination

• the expected calculations, issues, remediation, and reporting behavior

These scenarios should be understandable by Compliance, version-controlled, automatically executable, and tied directly to release readiness.

Your first 90 days

First 30 days

• Learn the architecture of the compliance engine and how our engineering and Compliance teams currently validate changes

• Map at least one major compliance framework into a structured set of scenarios

• Identify the highest-risk gaps in our existing automated coverage

By 60 days

• Build the initial version of the compliance scenario registry

• Automate a meaningful set of high-priority scenarios, including regulatory citations and expected outcomes

• Establish a repeatable process for turning production defects into regression tests

By 90 days

• Have critical compliance scenarios running automatically in CI

• Give Engineering and Compliance clear visibility into coverage, failures, and known gaps

• Establish release-readiness criteria that provide concrete evidence that changes to the compliance engine are safe to ship

Must have

• Strong experience in software QA, quality engineering, or test automation

• Strong Python skills

• Experience building automated integration and end-to-end test suites

• Experience integrating testing into CI/CD and release processes

• Ability to test a Python/React application in an AWS environment

• Ability to systematically reason about complex combinations of rules, inputs, and expected outcomes

• Ability to turn written requirements into precise, deterministic test cases

• Strong attention to traceability: not just “did this test pass?”, but “why is this the expected behavior, and what requirement supports it?”

• Ability to work closely with engineers and domain experts to resolve ambiguity

Nice to have

• Recent hands-on experience with Playwright

• Experience with QA Wolf or similar modern QA tools

• Experience with parameterized, property-based, combinatorial, or model-based testing

• Experience testing payroll, tax, financial, legal, healthcare, insurance, or other rules-heavy software

• Experience with requirements traceability, release certification, or audit evidence

What success looks like

• Dili has a structured inventory of the compliance scenarios we support

• Critical scenarios are automatically tested and tied to authoritative regulatory sources

• Compliance can review and contribute to expected behavior

• Production defects become permanent regression tests

• Changes to regulations or interpretations can be traced to affected scenarios

• Releases provide clear evidence of what was tested and whether the compliance engine is ready to ship