Senior Automation Tester in Python (for Gen AI powered applications)

EPAM Systems, Inc.; Polska; Hybrid; PLN 180,000 - 300,000; Full time

Benefits offered by this job

Health insurance

Hybrid by design

Relocation opportunities

Employee Stock Purchase Plan

Job summary

EPAM Systems, Inc. is seeking a Senior Automation Tester in Python to develop automated test frameworks for AI-powered clinical applications.

You will validate non-deterministic AI outputs and ensure GxP compliance through IQ/OQ/PQ qualification, working across Safety, Regulatory, Clinical Operations, and SAGE tooling. The role emphasizes building test strategies, integrating tests into CI/CD, and collaborating with data scientists and backend developers to define acceptance criteria and

Qualifications

• 5+ years of hands-on QA/test automation experience.
• Experience building automated test suites from scratch.
• Knowledge of CI/CD test integration and quality gates.

Responsibilities

• Build automated test frameworks from scratch for AI-powered clinical apps using Playwright, Selenium, and pytest.
• Design and execute test strategies for non-deterministic AI outputs, including accuracy and prompt regression.
• Integrate automated tests into CI/CD pipelines with block-on-failure and coverage thresholds.
• Execute IQ/OQ/PQ scripts and generate qualification evidence packages.
• Collaborate with Data Scientists and Backend Developers to define testable acceptance criteria.

Skills

Python

Test automation

API testing

CI/CD

AI/GenAI testing

Playwright

Selenium

Cypress

pytest

Jenkins

GitHub Actions

Postman/Newman

Locust

Tools

Playwright

Selenium

Cypress

pytest

Jenkins

GitHub Actions

Postman/Newman

Job description

We are looking for a Senior Automation Tester in Python to join Production Pods within the program. This is not a manual/functional QA role. You will build and maintain automated test frameworks for AI/GenAI-powered clinical applications, validate non-deterministic AI outputs, and ensure GxP compliance through automated IQ/OQ/PQ qualification. You will work across multiple use cases in Safety, Regulatory, Clinical Operations, and SAGE platform tooling.ResponsibilitiesBuild automated test frameworks from scratch for AI-powered clinical applications using Playwright, Selenium, and pytestDesign and execute test strategies for non-deterministic AI/GenAI outputs, including accuracy, RAG relevance, hallucination checks, and prompt regressionIntegrate automated tests into CI/CD pipelines with quality gates such as block-on-failure and coverage thresholdsExecute GxP-compliant IQ/OQ/PQ scripts and generate qualification evidence packagesManage AI-specific test data, including edge cases, adversarial inputs, and golden datasetsMonitor model drift, regression, and performance degradation over timeProduce test strategy documents, test plans, and compliance-ready test evidenceCollaborate with Data Scientists and Backend Developers to define testable acceptance criteria for AI featuresRequirements5+ years of hands-on QA/test automation experienceExpertise in building and maintaining automated test suites from scratch using Playwright, Selenium, Cypress, and pytestSkills in API and backend test automation, including REST APIs, microservices, and serverless architectures with Postman/Newman and pytest + requestsProficiency in Python as the primary automation language, with the ability to write harnesses, fixtures, and utilities independentlyKnowledge of CI/CD test integration, embedding automated tests into pipelines such as Jenkins and GitHub Actions with quality gatesBackground in testing AI/GenAI applications, validating non-deterministic AI/LLM outputs for accuracy, RAG relevance, hallucination, and prompt regressionFamiliarity with AI/ML model testing, including accuracy, bias, and performance validation using RAGAS, LLM-as-judge, and golden datasetsEnglish proficiency at B2 level or higherNice to haveBackground in pharmaceutical / life sciences domain, with GxP awareness and IQ/OQ/PQ qualificationExperience with Jira/Xray, TestRail, or Zephyr test managementSkills in performance/load testing using JMeter and LocustUnderstanding of security testing basics such as OWASPFamiliarity with AWS services including Lambda, API Gateway, S3, and CloudWatchWe offerWe gather like-minded people:Top tech minds driving innovation in AI, cloud and digital platform modernizationSupportive team and agile, startup-like cultureHybrid by design mode and opportunity to work remotely within PolandChance to work abroad for up to 60 days annuallyBusiness-driven relocation opportunitiesWe provide growth opportunities:Career development programsThought leadership, mentoring, soft skills and well-being programsCertification (Anthropic, Gemini, GCP, Azure, AWS)English classesWe cover it all:Stable payParticipation in the Employee Stock Purchase Plan with a 15% discountBenefits package (health insurance, multisport, shopping vouchers)Referral bonuses up to $2,000Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and moreCorporate, social and well-being eventsPlease, note:Benefits listed above are available to employees onlyWe are open for working with Contractors. Terms of B2B cooperation agreements are agreed individuallyWe will reach out to selected candidates exclusivelyEPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.