Senior Test Automation Engineer (Python)

Edvantis; Rzeszów; Remote; PLN 180,000 - 240,000; Full time

Job summary

A leading tech company is looking for a Senior Test Automation Engineer (Python) to design and develop automated test frameworks for AI-driven solutions. The role involves API testing, integrating test automation into CI/CD pipelines, and conducting manual exploratory testing when needed. Candidates should have over 4 years of experience in QA Automation and strong Python skills. This position offers a remote-first work model and competitive compensation packages including medical insurance and mental health programs.

Benefits

Remote-first work model

Medical insurance

18 paid vacation days

Mental health program

Language courses

Corporate subscription to learning platforms

Qualifications

• 4+ years of experience in QA Automation with hands-on skills in Python & Pytest.
• Experience testing APIs (REST, GraphQL).
• Knowledge of CI/CD pipelines, preferably GitHub Action.

Responsibilities

• Design, develop, and maintain automated test scripts and frameworks.
• Create test strategies for validating AI agent workflows.
• Integrate test automation into CI/CD pipelines.

Skills

QA Automation

Python

Pytest

API testing

Debugging skills

Problem-solving skills

Communication skills

Tools

Postman

GitHub Actions

Blazemeter

JMeter

Job description

We’re opening the position of Senior Test Automation Engineer (Python) to the new team in the project of our customers. They provide SAAS solutions that help companies optimize their businesses. These solutions include business planning & execution, process automation, and optimization. The Customer’s platform leverages industry-leading artificial intelligence (AI agents).
In this position, you will be responsible for the AI Agents testing through UI and API layers.

Responsibilities:

• Design, develop, and maintain automated test scripts and frameworks using Python and Pytest
• Create test strategies for validating AI agent workflows, APIs, and Agent integrations
• Implement end-to-end, integration, and regression testing for AI agent solutions
• Integrate test automation into CI/CD pipelines (GitHub Actions)
• Perform manual exploratory testing where automation is not yet feasible
• Build and execute test plans for validating:
• – Conversational logic and accuracy of AI agents
• – API interactions, data pipelines, and back-end services
• – Performance, reliability, and scalability of AI-driven services

Requirements:

• 4+ years of experience in QA Automation, with strong hands-on skills in Python & Pytest
• Experience testing APIs (REST, GraphQL) with tools like Postman/Newman
• Knowledge of CI/CD pipelines and automation integration (GitHub Action would be a plus)
• Strong debugging, troubleshooting, and problem-solving skills
• Familiarity with performance/load testing tools (Blazemeter, JMeter)
• Excellent written and spoken communication skills
• English level – Upper-Intermediate or higher

Will be a plus:

• Familiarity with AI/ML testing approaches (e.g., NLP testing, model response validation, fuzz testing) would be a plus
• Hands-on experience testing AI conversational agents (e.g., OpenAI, Azure OpenAI, Copilot, Dialogflow, Rasa, LangChain) would be a plus

We offer:

• Remote-first work model with flexible working hours (we provide all equipment)
• Comfortable and fully equipped offices in Lviv and Rzeszów
• Competitive compensation with regular performance reviews
• 18 paid vacation days per year + all state holidays
• 12 days of paid sick leave per year without a medical certificate + extra paid leave for blood donation
• Medical insurance with an affordable family coverage option
• Mental health program which includes free and confidential consultations with a psychologist
• English, German, and Polish language courses
• Corporate subscription to learning platforms, regular meetups and webinars
• Friendly team that values accountability, innovation, teamwork, and customer satisfaction
• Inclusive environment where everyone feels valued and treated equally. We proudly partner with VeteranHub to support Ukrainian veterans
• We are committed to supporting Ukraine and actively participate in charity initiatives