QA Engineer – AI Web App Automation & Reliability

Mercor; New York (NY); On-site; USD 100,000 - 140,000; Full time

Job summary

Mercor is seeking experienced Software QA Engineers, SDETs, and Test Automation Experts to review browser-based evaluation workflows for AI-driven web applications. You will validate test determinism, isolation, and reliability before benchmarks are produced, ensuring clear pass/fail outcomes.

Ideal candidates have 3+ years in software QA or test automation, strong web app testing experience, and proficiency with Playwright, Cypress, or Selenium.

Qualifications

• Minimum 3+ years in Software QA, SDET or test automation.
• Experience with modern web applications and UI tests.
• Strong debugging skills and attention to detail.

Responsibilities

• Review browser-based testing workflows for AI-generated web apps.
• Assess feasibility and reliability of browser interactions.
• Validate test isolation, fixtures, setup and execution flow.
• Ensure deterministic pass/fail results in tests.
• Identify flaky tests and ambiguous validation logic.
• Provide structured technical feedback against rubrics.

Skills

Software QA

Web Testing

Debugging

Test Automation

Attention to detail

Tools

Playwright

Cypress

Selenium

Job description

Mercor is seeking experienced Software QA Engineers, SDETs, and Test Automation Experts to review browser-based evaluation workflows for AI-driven web applications. You will validate test determinism, isolation, and reliability before benchmarks are produced, ensuring clear pass/fail outcomes.

Ideal candidates have 3+ years in software QA or test automation, strong web app testing experience, and proficiency with Playwright, Cypress, or Selenium.