Principal Quality Engineer (AI-Assisted & Agentic Testing)

Modus Create

United States

Remote

USD 150,000 - 210,000

Full time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Modus Create is seeking a Principal Quality Engineer to lead AI-assisted testing in complex enterprise environments. This hands-on role designes scalable automation, builds AI‑driven testing workflows, and accelerates test creation, regression testing, and quality validation across COBOL, Java/Spring, and modern frontends.

You will mentor QA engineers and developers, collaborate with client teams to raise automation coverage, improve regression cycle times, and ensure production‑grade quality in

Qualifications

  • 10+ years of professional experience in Quality Assurance, Test Automation, SDET, or related software engineering roles.
  • Strong hands-on experience designing and building test automation frameworks from scratch using Playwright or Cypress.
  • Advanced expertise in API testing, integration testing, end-to-end testing, and automated regression testing.
  • Hands-on experience using Claude Code, Claude CLI, GitHub Copilot, or AI coding agents to generate, review, refactor, and maintain automated tests.
  • Experience configuring agentic testing workflows, prompts, repository memory files, guidelines, and reusable AI-assisted testing patterns.
  • Proven experience converting manual regression suites into automated tests within existing enterprise applications.
  • Strong experience integrating automated tests into CI/CD pipelines, using GitHub Actions, Jenkins, or similar tools.
  • Solid understanding of test strategy, risk-based testing, test data management, coverage analysis, and flaky-test diagnosis.
  • Ability to critically evaluate AI-generated test cases and scripts for coverage, assumptions, and security risks.
  • Experience establishing guardrails and validation processes to ensure production-quality AI-generated tests.
  • Experience defining, tracking, and improving quality engineering metrics (automation coverage, test creation time, regression cycle time).
  • Familiarity with testing AI-powered applications, including evaluating LLM outputs for accuracy and non-determinism.
  • Understanding of Agentic AI Evaluations (Evals) and evaluation datasets, scoring criteria, reliability measurements, and human review processes.
  • Demonstrated experience mentoring QA engineers and developers through hands-on pairing and collaboration.
  • Working knowledge of Git, Jira, and test management platforms such as Xray, TestRail, or Zephyr.
  • Exposure to COBOL, mainframe, legacy enterprise applications, or financial services systems.

Responsibilities

  • Lead design and implementation of AI-assisted quality engineering practices across enterprise environments.
  • Develop scalable automation frameworks and AI-driven testing workflows to accelerate test creation and regression testing.
  • Mentor QA engineers and developers through hands-on pairing and knowledge sharing.
  • Collaborate with client engineering teams to improve testing coverage and quality across COBOL, Java/Spring, and modern frontends.

Skills

QA leadership
Test automation
SDET
API testing
End-to-end testing
CI/CD
AI-assisted testing
Mentoring
Java/Spring
TypeScript/React

Tools

Playwright
Cypress
Claude Code
Claude CLI
GitHub Copilot
GitHub Actions
Jenkins
Xray
TestRail
Zephyr

Job description

Principal Quality Engineer (AI-Assisted & Agentic Testing)
About Us

Founded in 2011, Modus is a global, fully remote team of world‑class technologists who thrive in a collaborative, innovative environment. We’re a digital product engineering partner for forward‑thinking businesses. Our global teams work side‑by‑side with clients to design, build, and scale custom solutions that achieve real results and lasting change, partnering with industry leaders including AWS, GitHub, and Atlassian. We were fully remote before it was cool! Recognized as one of the Inc. 5000 Fastest Growing Private Companies for nine years and a top remote work company by FlexJobs, we have helped some of the world’s largest brands deliver powerful digital experiences. As an award‑winning Atlassian partner with a world‑class team, we help organizations innovate and solve complex challenges for Fortune 500 companies and beyond, we want to hear from you.

Opportunity! We are looking for a Principal Quality Engineer to help transform quality engineering practices by introducing AI-assisted testing, agentic automation, and modern quality engineering frameworks within complex enterprise environments. You'll work directly within a client's engineering team, assessing existing testing practices, building scalable automation solutions, and leveraging Claude Code and AI coding agents to accelerate test creation, regression testing, and quality validation. This is a highly hands‑on engineering and consulting role that combines advanced test automation expertise with agentic AI development practices. You'll contribute directly to production testing initiatives, develop reusable AI‑assisted testing workflows, and mentor QA engineers and developers through practical collaboration and hands‑on pairing. You'll also work across enterprise applications involving COBOL systems, Java/Spring middleware, and TypeScript/React frontends, helping ensure quality across integrated, mission‑critical systems.

Requirements
  • 10+ years of professional experience in Quality Assurance, Test Automation, Software Development Engineering in Test (SDET), or related software engineering roles.
  • Strong hands‑on experience designing and building test automation frameworks from scratch, using tools such as Playwright, Cypress, or similar technologies.
  • Advanced expertise in API testing, integration testing, end‑to‑end testing, and automated regression testing.
  • Hands‑on experience using Claude Code, Claude CLI, GitHub Copilot, or AI coding agents to generate, review, refactor, and maintain automated tests.
  • Experience configuring agentic testing workflows, including prompts, custom skills, repository memory files, guidelines, and reusable AI‑assisted testing patterns.
  • Proven experience converting manual regression suites into automated tests, particularly within existing enterprise applications.
  • Strong experience integrating automated tests into CI/CD pipelines, using GitHub Actions, Jenkins, or similar tools.
  • Solid understanding of test strategy, risk‑based testing, test data management, coverage analysis, and flaky‑test diagnosis.
  • Ability to critically evaluate AI‑generated test cases and scripts, identifying incomplete coverage, incorrect assumptions, unreliable assertions, and potential security risks.
  • Experience establishing guardrails and validation processes to ensure AI‑generated tests meet production‑quality standards, including secure handling of credentials, dependencies, and sensitive data.
  • Experience defining, tracking, and improving quality engineering metrics, such as automation coverage, test creation time, regression cycle time, flaky‑test rates, and escaped defects.
  • Familiarity with testing AI‑powered applications, including evaluating LLM‑generated outputs for accuracy, consistency, hallucinations, and non‑deterministic behavior.
  • Understanding of Agentic AI Evaluations (Evals), including evaluation datasets, scoring criteria, reliability measurements, and human review processes.
  • Demonstrated experience mentoring and upskilling QA engineers and developers, particularly through hands‑on pairing and collaborative development.
  • Working knowledge of Git, Jira, and test management platforms such as Xray, TestRail, or Zephyr.
  • Exposure to COBOL, mainframe, legacy enterprise applications, or financial services systems, including lending or account servicing environments.
  • Strong communication and consulting skills, with the ability to collaborate with engineering teams and stakeholders to introduce practical improvements in quality engineering.
Bonus points
  • Experience testing COBOL‑based systems of record and applications integrated with Java/Spring and modern frontend technologies.
  • Experience building custom Claude Code agents, testing skills, prompt libraries, and reusable AI‑assisted QA toolkits.
  • Familiarity with LLM evaluation frameworks, automated agent evaluations, and systematic validation of AI‑generated outputs.
  • Experience extracting complex business rules from legacy applications to develop accurate test scenarios and expected results.
  • Experience using AI‑assisted tools for test data generation, failure triage, regression test selection, and automated defect analysis.
  • Background in financial services, lending, account servicing, or other highly regulated enterprise environments.
  • Experience introducing AI‑assisted quality engineering practices across multiple engineering teams.
  • Experience developing QA playbooks, technical documentation, and engineering enablement programs.
  • Experience measuring the impact of AI‑assisted testing on engineering productivity, software quality, and delivery efficiency.
You’ll Love
  • Building production‑ready test automation frameworks that improve software quality and engineering efficiency.
  • Using Claude Code and agentic AI tools to accelerate test generation, regression testing, and failure analysis.
  • Transforming manual testing processes into scalable, AI‑assisted automation workflows.
  • Working across complex enterprise architectures involving COBOL, Java/Spring, and modern web applications.
  • Mentoring QA engineers and developers through hands‑on pairing, shared test development, and practical knowledge sharing.
  • Designing reusable AI testing toolkits, custom agent skills, prompts, and validation guardrails.
  • Having the autonomy to define testing strategies, tooling configurations, and quality engineering improvements.
  • Seeing measurable improvements in test coverage, regression cycle times, and engineering team capabilities.

By joining our team, you'll be part of a winning squad that plays to each other's strengths and celebrates every success together.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Principal Software Engineer (COBOL & Agentic Systems Enablement)
Principal Software Engineer (COBOL & Agentic Systems Enablement)

Modus Create • United States

Remote
USD 180,000 - 260,000
Senior QA Engineer
Senior QA Engineer

Intetics • United States

Remote
USD 120,000 - 180,000
Quality Assurance Engineer
Quality Assurance Engineer

AIM Qualifications and Assessment Group • United States

Remote
USD 90,000 - 130,000
Fully remote work environment
International client exposure
Certification & professional dev
Principal Full Stack Engineer (Agentic Systems & Enablement)
Principal Full Stack Engineer (Agentic Systems & Enablement)

Modus Create • United States

Remote
USD 150,000 - 230,000
1214 | Senior Agentic QA Engineer
1214 | Senior Agentic QA Engineer

Intetics • United States

Remote
USD 150,000 - 210,000
1214 | Senior Agentic QA Engineer
1214 | Senior Agentic QA Engineer

Intetics • Naples (FL)

On-site
USD 120,000 - 170,000
Senior Staff SDET (Customer Engineering)
Senior Staff SDET (Customer Engineering)

Hims & Hers Health, Inc. • United States

Remote
USD 180,000 - 240,000
Generous PTO
Full healthcare
401(k) with matching
+1
Senior QA Engineer
Senior QA Engineer

Intercontinental Exchange Holdings • Atlanta (GA)

On-site
USD 110,000 - 150,000
AI-Augmented QA Engineer
AI-Augmented QA Engineer

Pavago • United States

Remote
USD 90,000 - 140,000
Software Engineer in Test (AI-Driven Functional and Automation)
Software Engineer in Test (AI-Driven Functional and Automation)

Groupe Kardol • Philadelphia

Hybrid
USD 130,000 - 180,000