QA Engineer for AI-powered Salesforce Apps | Remote

Ascendix; Kraków; On-site; PLN 140,000 - 210,000; Full time

Benefits offered by this job

20 paid days-off

5 paid sick days

B2B and remote cooperation

Career development and professional成长

Technical performance reviews and 360°

External event compensation

Internal knowledge-sharing

Online and offline events and teamings

Open management

Job summary

Ascendix Technologies is seeking an experienced QA Engineer to own quality assurance for Salesforce-related products, starting with the Ascendix AI Agent. You will combine manual testing, test automation, and LLM evaluation techniques to validate AI-driven features and ensure reliability across both traditional software and AI-enabled capabilities.

The role requires 5+ years in QA, hands-on AI testing experience, and strong API testing skills.

Qualifications

• 5+ years of experience in Software Quality Assurance.
• Hands-on experience testing AI-powered applications with non-deterministic outcomes, such as LLM-based products, AI agents, or chatbots.
• Practical experience with LLM evaluation techniques, including test datasets, scoring rubrics, LLM-as-judge approaches, and regression tracking.
• Experience in test automation and maintaining automated test suites.
• Solid API testing skills using Postman or similar tools.
• Strong analytical and critical thinking skills, with the ability to evaluate not only whether an AI-generated response is incorrect, but also why.
• Ownership mindset, proactive approach to problem-solving, and willingness to improve QA processes.
• Good communication and collaboration skills, with the ability to work effectively in a small, distributed team.
• Upper-Intermediate English (B2+) or higher, both written and spoken.

Responsibilities

• Take ownership of product quality across our Salesforce-related solutions, initially focusing on the Ascendix AI Agent.
• Perform functional, regression, and exploratory testing of both deterministic and non-deterministic functionality.
• Evaluate AI-generated responses for accuracy, hallucinations, and correct execution of actions.
• Design, maintain, and improve LLM evaluation datasets, scoring criteria, and regression tests.
• Develop and maintain automated tests using Playwright and modern AI-assisted tools.
• Collaborate with developers and the Business Analyst in a spec-driven development process, reviewing specifications and defining clear acceptance criteria.
• Establish and continuously improve QA processes, including maintaining AI agent skills, rules, and workflow documentation in Markdown files.
• Investigate and reproduce issues reported by clients, providing clear and actionable bug reports.
• Proactively identify quality risks and suggest improvements to product reliability and testing practices.

Skills

Software QA

AI-powered testing

LLM evaluation

Test automation

API testing

Analytical thinking

Ownership mindset

Communication skills

English (B2+)

Tools

Postman

Playwright (TS/JS)

Promptfoo

Langfuse

DeepEval

Claude Code

Job description

Ascendix Technologies is seeking an experienced QA Engineer to own quality assurance for Salesforce-related products, starting with the Ascendix AI Agent. You will combine manual testing, test automation, and LLM evaluation techniques to validate AI-driven features and ensure reliability across both traditional software and AI-enabled capabilities.

The role requires 5+ years in QA, hands-on AI testing experience, and strong API testing skills.