BigData Test Automation Specialist

EPAM Systems; Zürich; On-site; CHF 120,000 - 180,000; Full time

Benefits offered by this job

5 weeks of vacation

EPAM Employee Stock Purchase Plan (ESo

Enhanced parental leave

Extended pension plan

Daily sickness allowance insurance

Employee assistance program

Training & certifications

Job summary

EPAM Systems, Inc. in Zurich is seeking a BigData Test Automation Specialist to design and implement a robust testing strategy for a modern big data platform in the banking sector.

You will work with Trino, Kafka, Iceberg and Dataiku to ensure data quality, platform resilience and system integrity. The role focuses on developing scalable test frameworks, validating streaming pipelines, and integrating automated Dataiku scenarios into CI/CD pipelines, while building resilient test infra and

Qualifications

• Experience building scalable test automation frameworks for big data.
• Ability to design and implement data quality checks for schemas.
• Hands-on with streaming data pipelines using Kafka.
• Experience validating Hive metastore schemas and data.
• Strong Python development and Pytest experience.
• Familiarity with CI/CD pipelines and containerization.

Responsibilities

• Design and implement comprehensive test automation strategies for a big data platform.
• Develop and maintain scalable test frameworks.
• Implement declarative data quality checks for schemas.
• Create and manage automated tests for complex SQL transformations.
• Test and validate Hive metastore schemas and data integrity.
• Develop tests for streaming data pipelines using Kafka and Testcontainers.
• Automate Dataiku DSS scenarios and integration into CI/CD pipelines.
• Build and optimize CI/CD infrastructure for efficient big data testing.
• Perform resilience and chaos testing to ensure platform stability.
• Use data versioning patterns like Write-Audit-Publish for isolated and safe testing.

Skills

Python

Pytest

Test automation

Big data

CI/CD

SQL

Kafka

Trino

Hive

Data quality

Dataiku

Docker

Testcontainers

S3

dbt

SQLMesh

Data versioning

Tools

Docker

Testcontainers

GitHub Actions

GitLab CI

Dataiku

dbt

SQLMesh

Kafka

Trino

Hive

S3

Job description

We are looking for a BigData Test Automation Specialist to join our team in Zurich. This role focuses on designing and implementing a robust, multi-layered testing strategy for a modern big data platform within the banking sector. You will work with cutting-edge technologies including Trino, Kafka, Iceberg and Dataiku to ensure data quality, platform resilience and overall system integrity.ResponsibilitiesDesign and implement comprehensive test automation strategies for a big data platformDevelop and maintain scalable test frameworksImplement declarative data quality checks for schemasCreate and manage automated tests for complex SQL transformationsTest and validate Hive metastore schemas and data integrityDevelop tests for streaming data pipelines using Kafka and TestcontainersAutomate Dataiku DSS scenarios and integrate them into CI/CD pipelinesBuild and optimize CI/CD infrastructure for efficient big data testingPerform resilience and chaos testing to ensure platform stabilityUse data versioning patterns like Write-Audit-Publish for isolated and safe testingRequirementsStrong experience with Python and the Pytest framework for test automationHands-on experience with big data technologies like Trino, Hive, Kafka or S3Proficiency with data quality and validation toolsExperience with containerization and mocking tools like Docker, TestcontainersProven ability to test streaming applications and event-driven architecturesKnowledge of scripting for cloud object storage interactionExperience with MLOps platforms, preferably DataikuProven experience building and managing CI/CD pipelines (e.g. GitHub Actions, GitLab CI)Familiarity with SQL transformation tools like dbt or SQLMeshUnderstanding of data versioning conceptsFluent in EnglishNice to haveExperience in the banking or financial services sectorWe offer5 weeks of vacationEPAM Employee Stock Purchase Plan (ESPP)Enhanced parental leaveExtended pension planDaily sickness allowance insuranceEmployee assistance programGlobal business travel medical and accident insuranceLearning and development opportunities including in-house training and coaching, professional certifications, and courses• Please note that any offers will be subject to appropriate background checks• We do not accept CVs from recruiting or staffing agencies• For this position, we are able to consider applications from the following:- Swiss nationals- EU/EFTA nationals- Third-country nationals based in Switzerland with an appropriate work permit- Displaced people from Ukraine who are currently in Switzerland and hold, or have already applied for, S permits