QA Engineer

Ascendix; Kraków; On-site; PLN 120,000 - 180,000; Full time

Benefits offered by this job

20 paid days-off

5 paid sick days

B2B and remote cooperation

Career development and growth

360 Evaluation

Job summary

Ascendix Technologies, a fast-growing AI and Real Estate technology partner, seeks an experienced QA Engineer to own quality for Salesforce-related products, beginning with Ascendix AI Agent. You will blend manual testing, automation, and LLM evaluation to ensure reliable AI-driven features and traditional software functions.

You’ll work in a small, distributed team, shaping QA processes, and collaborating with developers and business analysts to define clear acceptance criteria.

Qualifications

• 5+ years in Software Quality Assurance.
• Experience testing AI-powered apps with non-deterministic outcomes.
• Experience with LLM evaluation techniques and regression tracking.
• Experience in test automation and maintaining automated suites.
• Strong API testing skills (Postman or similar).
• Analytical, able to explain why AI outputs are correct/incorrect.
• Ownership mindset and proactive QA process improvement.
• Good communication in a distributed team.
• Upper-Intermediate English (B2+) or higher.

Responsibilities

• Own product quality for Salesforce-related solutions, starting with Ascendix AI Agent.
• Perform functional, regression, and exploratory testing of deterministic and non-deterministic features.
• Evaluate AI-generated responses for accuracy and hallucinations.
• Develop and maintain LLM evaluation datasets and regression tests.
• Build and maintain automated tests using Playwright and AI-assisted tools.
• Collaborate with developers and Business Analyst to define acceptance criteria.
• Improve QA processes and document AI agent skills and workflows.
• Investigate and reproduce client-reported issues and provide actionable bug reports.

Skills

QA

AI/LLM testing

Test automation

Postman

Playwright

Critical thinking

Communication

English (B2+)

Tools

Postman

Playwright

Job description

Ascendix Technologies is an AI and Real Estate Technology partner headquartered in Dallas, Texas, with offices in Poland, Portugal, and Ukraine. We specialize in AI solutions, Salesforce, and custom software development, helping businesses streamline their operations through innovative technology.

We're looking for an experienced QA Engineer to join our Product Department and take ownership of quality assurance for our Salesforce-related products, starting with the Ascendix AI Agent.

In this role, you'll work with a small, collaborative team to ensure the reliability of both traditional software functionality and AI-driven features. You'll combine manual testing, test automation, and LLM evaluation techniques to validate AI-generated responses, identify potential issues, and continuously improve product quality.

If you enjoy exploring AI technologies, thinking critically about complex scenarios, and taking initiative in shaping QA processes, we'd love to hear from you.

What We're Looking For

Requirements:

• 5+ years of experience in Software Quality Assurance;
• Hands-on experience testing AI-powered applications with non-deterministic outcomes, such as LLM-based products, AI agents, or chatbots;
• Practical experience with LLM evaluation techniques, including test datasets, scoring rubrics, LLM-as-judge approaches, and regression tracking;
• Experience in test automation and maintaining automated test suites;
• Solid API testing skills using Postman or similar tools;
• Strong analytical and critical thinking skills, with the ability to evaluate not only whether an AI-generated response is incorrect, but also why;
• Ownership mindset, proactive approach to problem-solving, and willingness to improve QA processes;
• Good communication and collaboration skills, with the ability to work effectively in a small, distributed team;
• Upper-Intermediate English (B2+) or higher, both written and spoken.

Nice to Have:

• Experience with Playwright (TypeScript or JavaScript);
• Salesforce testing experience, including Lightning, managed packages, and permission sets;
• Familiarity with LLM evaluation tools such as Promptfoo, DeepEval, Ragas, or Langfuse;
• Experience using AI-assisted development tools such as Claude Code, Cursor, or similar;
• Basic prompt engineering skills;
• Experience working with CRM systems or real estate data.

What You'll Do

Responsibilities:

• Take ownership of product quality across our Salesforce-related solutions, initially focusing on the Ascendix AI Agent;
• Perform functional, regression, and exploratory testing of both deterministic and non-deterministic functionality;
• Evaluate AI-generated responses for accuracy, hallucinations, and correct execution of actions;
• Design, maintain, and improve LLM evaluation datasets, scoring criteria, and regression tests;
• Develop and maintain automated tests using Playwright and modern AI-assisted tools;
• Collaborate with developers and the Business Analyst in a spec-driven development process, reviewing specifications and defining clear acceptance criteria;
• Establish and continuously improve QA processes, including maintaining AI agent skills, rules, and workflow documentation in Markdown files;
• Investigate and reproduce issues reported by clients, providing clear and actionable bug reports;
• Proactively identify quality risks and suggest improvements to product reliability and testing practices.

Ascendix Offers You:

Healthcare:

• 20 paid days-off;
• 5 paid sick days.

Career and Professional Growth:

• B2B and remote cooperation;
• Career development and professional growth;
• Technical performance reviews and 360 Evaluation;
• Collaboration with a personal Direct Manager, Technical Leads, and mentors.

Learning and Development:

• Opportunity for external event compensation;
• Internal knowledge-sharing opportunities.

Corporate Spirit:

• Online and offline events and team buildings;
• Friendly and supportive atmosphere;
• Open management.