AI-Driven QA & Automation Engineer for LLM/ML Testing

Robotics Prcocess Automation, LLC; Austin (TX); On-site; USD 55,000 - 117,000; Full time

Job summary

Robotics Process Automation, LLC is looking for a QA & Automation Engineer with experience in AI tools to develop test suites for multi-agent workflows and APIs in Austin, TX. The role involves automating regression tests for LLM-driven features. Candidates must have a strong background in automation for AI/LLM systems and proficiency in API/E2E testing using tools like Playwright and Cypress. This position offers an hourly rate depending on experience and may include extensions beyond the initial 12 months.

Qualifications

• Experience with test automation in AI/LLM systems, including flakiness and hallucinations.
• Proficiency in API/E2E testing using Playwright, Cypress, and Pytest.
• Familiarity with agent reliability evaluation frameworks.

Responsibilities

• Develop AI-aligned test suites for multi-agent workflows and APIs.
• Work with Test Agent to auto-generate and optimize test coverage.
• Automate regression tests for LLM-driven features.

Skills

AI-aligned test suites

LLM system

ML model testing

API/E2E testing (Playwright, Cypress, Pytest)

Job description

Robotics Process Automation, LLC is looking for a QA & Automation Engineer with experience in AI tools to develop test suites for multi-agent workflows and APIs in Austin, TX. The role involves automating regression tests for LLM-driven features. Candidates must have a strong background in automation for AI/LLM systems and proficiency in API/E2E testing using tools like Playwright and Cypress. This position offers an hourly rate depending on experience and may include extensions beyond the initial 12 months.