QA Engineer (AI & Feature Testing) (auto)

Solvd; United States; Remote; USD 90,000 - 150,000; Full time

Job summary

Solvd is seeking a Senior QA Engineer to join a team building and validating AI-powered and traditional features for a leading personal finance platform. You’ll own quality across web, mobile, and API with a focus on testing AI-integrated components where deterministic methods don’t apply.

The role requires hands-on Playwright testing, involvement in eval-based assessments, and collaboration across global teams.

Qualifications

• 5–7 years of experience in software testing across web, mobile, and API.
• Hands-on experience with Playwright in a production testing environment.
• Hands-on AI feature testing experience validating AI/LLM-integrated products.
• Clear understanding of eval vs. deterministic testing approaches.
• Strong understanding of web and mobile app lifecycles including iOS/Android specifics.

Responsibilities

• Plan, design, and execute test cases for AI-powered and traditional features.
• Analyze traditional requirements and AI model behavior specifications.
• Create comprehensive test plans, test cases, and scripts from business requirements.
• Perform manual feature testing across platforms and verify bug fixes.
• Document defect reports and AI model responses, hallucinations, and regressions.

Skills

Manual testing

QA testing

Analytical thinking

Problem solving

Education

Bachelor's degree in Computer Science

Tools

Playwright

Jira

Confluence

HTML proxy tools

Fiddler

Charles Proxy

Xcode logs

ADB logs

Test management

Job description

Описание: Solvd Inc. is a rapidly growing AI-native consulting and technology services firm delivering enterprise transformation across cloud, data, software engineering, and artificial intelligence. We work with industry-leading organizations to design, build, and operationalize technology solutions that drive measurable business outcomes. Following the acquisition of Tooploox, a premier AI and product development company, Solvd now offers true end-to-end delivery—from strategic advisory and solution design to custom AI development and enterprise-scale implementation. Our capability centers combine deep technical expertise, proven delivery methodologies, and sector-specific knowledge to address complex business challenges quickly and effectively. We are looking for a Senior QA Engineer to join a team building and validating AI-powered and traditional features for a leading personal finance platform. You’ll own quality across web, mobile, and API — with a particular focus on testing AI/LLM-integrated components where standard deterministic methods don’t apply. This role requires someone who understands the fundamental difference between testing software that has a right answer and evaluating AI outputs that have a probability distribution — and who has done both in production.

Задачи:

• Plan, design, and execute test cases for AI-powered components and traditional features, applying the appropriate methodology — eval-based or deterministic — based on the target behavior.
• Analyze traditional functional requirements as well as AI model behavior specifications, prompt outputs, and edge-case scenarios.
• Turn business requirements, user flows, and business process procedures into comprehensive test plans, test cases, and test scripts.
• Perform manual feature testing across platforms and verify bug fixes.
• Create clear, detailed defect reports and document unexpected AI model responses, hallucinations, and performance regressions.
• Assist in developing QA practices for emerging technologies including AI frameworks, and disseminate best practices across the team.
• Become a subject matter expert in core systems, interfaces, business processes, and reporting tools.
• Partner closely with functional teams, engineers, and IT in a global environment with aligned LATAM working hours.
• Shape real-world AI-driven projects across key industries, working with clients from startup innovation to enterprise transformation.
• Be part of a global team with equal opportunities for collaboration across continents and cultures.
• Thrive in an inclusive environment that prioritizes continuous learning, innovation, and ethical AI standards.

Требования:

• 5–7 years of experience in software testing across web, mobile, and API.
• Hands-on experience with Playwright in a production testing environment — this is a primary requirement for this role.
• Hands-on AI feature testing experience — validating AI/LLM-integrated products for accuracy, relevance, robustness against adversarial prompts, edge cases, and safety/toxicity.
• Clear understanding of eval vs. deterministic testing — the difference between pass/fail based on exact input/output and evaluating probabilistic AI outputs using human-in-the-loop evaluations, score metrics, benchmark datasets, and semantic evaluation.
• Strong understanding of web and mobile application development lifecycles including iOS and Android specifics.
• Proven experience in integration, regression, performance/load, and UAT.
• Hands-on knowledge of browser DevTools and log monitoring.
• Advanced proficiency in Jira and Confluence.
• Experience with test management systems.
• Experience with HTML proxy tools such as Charles Proxy or Fiddler.
• Hands-on knowledge of Xcode logs and ADB logs.
• Experience in the financial services or fintech domain.
• Bachelor’s degree in Computer Science or equivalent.

Ready to make an impact? If you're excited to build things that matter, champion responsible AI, and grow with some of the industry’s sharpest minds.

Solvd is an equal opportunity employer.