Java QA Automation Engineer – AI/LLM & Playwright Focus

Qualysoft; United States; Remote; USD 90,000 - 120,000; Full time

Benefits offered by this job

Premium medical package

Lunch Tickets & Pluxee Card

Bookster subscription

13th salary/yearly bonuses

Flexible working program and remote-OK

Job summary

Qualysoft seeks an experienced QA automation engineer to fortify production-grade test coverage across Java backends and frontend interfaces. You will design and run Playwright/Cypress/Selenium E2E suites, implement robust API testing pipelines (REST, schema validation, auth/error scenarios), and ensure data integrity with SQL/PostgreSQL tests.

Responsibilities include building mocking strategies, setting up CI quality gates, and expanding coverage to untested UI/workflows.

Qualifications

• 3+ years in QA automation for production systems.
• Strong Java backend testing skills.
• Strong frontend testing.
• Experience building and maintaining Playwright/Cypress/Selenium E2E suites.
• API testing experience (REST, schema validation, auth/error scenarios).
• Experience with SQL/PostgreSQL testing and test data management.
• CI/CD pipeline experience (GitHub Actions, Azure Pipelines, or Jenkins).
• Practical understanding of testing non-deterministic LLM outputs.

Responsibilities

• Build API and integration test coverage for Java/Spring routers and database workflows.
• Design and implement Playwright E2E tests for critical user journeys.
• Create stable mocking and fixture strategies (MSW, test fixtures, seeded test data).
• Set up CI quality gates for linting, unit/integration/E2E tests, coverage, and reporting.
• Expand coverage to untested UI and workflow areas and prevent regressions.
• Build an LLM evaluation discipline with golden datasets and measurable quality gates.
• Track model output reliability, detect drift, and gate releases on quality/stability thresholds.
• Improve test maintainability, remove flakiness, and optimize runtime.
• LLM QA Tooling
• Use LangChain4j for JVM-native LLM application support, including agents, RAG, and guardrails.
• Use Langfuse Java SDK and OpenTelemetry for trace capture, prompt/version tracking, and experiment comparison.
• Use the OpenAI Java SDK for structured outputs, function calling, webhook verification, and response validation.
• Use Promptfoo in CI when you need language-agnostic regression and red-team checks around the Java stack.
• Use Java schema validation and guardrail patterns with Jackson, Swagger annotations, and LangChain4j guardrails.
• Use OpenTelemetry-based traces and Langfuse datasets as the reproducible source of truth for release gates.

Skills

Java backend testing

Frontend testing

API testing

REST API testing

SQL testing

PostgreSQL testing

CI/CD pipelines

LLM output testing

Playwright

Cypress

Selenium

Tools

Playwright

Cypress

Selenium

Job description

Qualysoft seeks an experienced QA automation engineer to fortify production-grade test coverage across Java backends and frontend interfaces. You will design and run Playwright/Cypress/Selenium E2E suites, implement robust API testing pipelines (REST, schema validation, auth/error scenarios), and ensure data integrity with SQL/PostgreSQL tests.

Responsibilities include building mocking strategies, setting up CI quality gates, and expanding coverage to untested UI/workflows.