Senior QA Engineer: AI-Driven Features Testing

OpenKrill; United States; A distancia; USD 120.000 - 160.000; Jornada completa

Descripción de la vacante

Solvd Inc. is seeking a Senior QA Engineer to own quality across web, mobile, and API for AI-powered and traditional features in a leading personal finance platform. You will design test plans, execute cases, and document AI outputs, including hallucinations and performance regressions, while collaborating with engineers and IT across a global team.

A strong background in Playwright, AI testing, and cross-platform testing is essential for success in this role.

Formación

• 5–7 years of experience in software testing across web, mobile, and API.
• Hands-on experience with Playwright in production testing.
• Hands-on AI feature testing experience — validating AI/LLM-integrated products for accuracy, relevance, robustness against adversarial prompts, edge cases, and safety/toxicity.
• Clear understanding of eval vs. deterministic testing — the difference between pass/fail based on exact input/output and evaluating probabilistic AI outputs using human-in-the-loop evaluations, score metrics, benchmark datasets, and semantic evaluation.
• Strong understanding of web and mobile application development lifecycles including iOS and Android specifics.
• Proven experience in integration, regression, performance/load, and UAT.
• Hands-on knowledge of browser DevTools and log monitoring.
• Advanced proficiency in Jira and Confluence.
• Experience with test management systems.
• Bachelor’s degree in CS or equivalent.

Responsabilidades

• Plan, design, and execute test cases for AI-powered components and traditional features, applying the appropriate methodology — eval-based or deterministic — based on the target behavior.
• Analyze traditional functional requirements as well as AI model behavior specifications, prompt outputs, and edge-case scenarios.
• Turn business requirements, user flows, and business process procedures into comprehensive test plans, test cases, and test scripts.
• Perform manual feature testing across platforms and verify bug fixes.
• Create clear, detailed defect reports and document unexpected AI model responses, hallucinations, and performance regressions.
• Assist in developing QA practices for emerging technologies including AI frameworks, and disseminate best practices across the team.
• Become a subject matter expert in core systems, interfaces, business processes, and reporting tools.
• Partner closely with functional teams, engineers, and IT in a global environment with aligned LATAM working hours.

Conocimientos

Software testing

AI testing awareness

Deterministic vs eval testing

Web/mobile testing

Regression testing

Educación

Bachelor’s degree in CS or equivalent

Herramientas

Playwright

Jira

Confluence

Test management systems

DevTools

Descripción del empleo

Solvd Inc. is seeking a Senior QA Engineer to own quality across web, mobile, and API for AI-powered and traditional features in a leading personal finance platform. You will design test plans, execute cases, and document AI outputs, including hallucinations and performance regressions, while collaborating with engineers and IT across a global team.

A strong background in Playwright, AI testing, and cross-platform testing is essential for success in this role.