QA Automation Engineer: AI & Web Testing

Chatoly Assistant; United States; Remote; USD 90,000 - 130,000; Full time

Job summary

Chatoly Assistant is seeking a QA automation engineer to own end-to-end web testing across multiple surfaces, including AI response evaluation considerations. You will build and maintain tests for the dashboard, widget, and public site, and design regression strategies with meaningful correctness criteria beyond fixed strings.

You will also validate integrations in partner sandboxes, perform cross-browser and accessibility checks, and seek speed and stability improvements to keep the suite

Qualifications

• At least three years of QA automation experience for web applications.
• Practical experience with Playwright, Cypress, or a comparable browser automation framework.
• Ability to read and write application code, not only test-specific scripts.
• Methodical debugging and defect-isolation habits.
• Sound judgment about which scenarios benefit from automation and which do not.

Responsibilities

• Develop and maintain end-to-end tests for the dashboard, embedded widget, and public-facing site.
• Design regression strategies for AI-generated responses using meaningful correctness criteria.
• Exercise integrations in partner sandboxes and identify compatibility problems before customers
• Include desktop, mobile, cross-browser, and accessibility checks in release process.
• Improve test execution speed, stability, and diagnostic value for team trust.
• Reproduce, isolate, and document defects clearly to support efficient fixes.

Skills

QA automation

Web application testing

Code reading/writing

Debugging

Automation strategy

Tools

Playwright

Cypress

Job description

Chatoly Assistant is seeking a QA automation engineer to own end-to-end web testing across multiple surfaces, including AI response evaluation considerations. You will build and maintain tests for the dashboard, widget, and public site, and design regression strategies with meaningful correctness criteria beyond fixed strings.

You will also validate integrations in partner sandboxes, perform cross-browser and accessibility checks, and seek speed and stability improvements to keep the suite