Big Data Test Automation Engineer — CI/CD Focus

EPAM Systems; Zürich; On-site; CHF 120,000 - 180,000; Full time

Benefits offered by this job

5 weeks vacation

EPAM Employee Stock Purchase Plan

Enhanced parental leave

Extended pension plan

Daily sickness allowance insurance

Employee assistance program

Travel medical/accident insurance

Learning and development opportunities

Background checks

Job summary

EPAM Systems in Zurich seeks a BigData Test Automation Specialist to design and implement a robust testing strategy for a modern big data platform in the banking sector. You will work with Trino, Kafka, Iceberg and Dataiku to ensure data quality and system integrity.

Responsibilities include building scalable test frameworks, automating CI/CD pipelines, testing streaming pipelines, and performing resilience testing to keep the platform stable and reliable.

Qualifications

• Strong experience with Python and the Pytest framework for test automation.
• Hands-on experience with Trino, Hive, Kafka or S3 for big data testing.
• Proficiency with data quality and validation tools.
• Experience with containerization and mocking tools like Docker, Testcontainers.
• Proven ability to test streaming applications and event-driven architectures.
• Knowledge of scripting for cloud object storage interaction.
• Experience with MLOps platforms, preferably Dataiku.
• Proven experience building and managing CI/CD pipelines (e.g. GitHub Actions, GitLab CI).
• Familiarity with SQL transformation tools like dbt or SQLMesh.
• Understanding of data versioning concepts.
• Fluent in English.

Responsibilities

• Design and implement comprehensive test automation strategies for a big data platform
• Develop and maintain scalable test frameworks
• Implement declarative data quality checks for schemas
• Create and manage automated tests for complex SQL transformations
• Test and validate Hive metastore schemas and data integrity
• Develop tests for streaming data pipelines using Kafka and Testcontainers
• Automate Dataiku DSS scenarios and integrate them into CI/CD pipelines
• Build and optimize CI/CD infrastructure for efficient big data testing
• Perform resilience and chaos testing to ensure platform stability
• Use data versioning patterns like Write-Audit-Publish for isolated and safe testing

Skills

Python

Pytest

Docker

Testcontainers

Kafka

Trino

Hive

S3

Dataiku

CI/CD

SQL

Data quality checks

English

Tools

Dataiku

dbt

SQLMesh

GitHub Actions

GitLab CI

Trino

Hive

Kafka

Testcontainers

Docker

S3

Job description

EPAM Systems in Zurich seeks a BigData Test Automation Specialist to design and implement a robust testing strategy for a modern big data platform in the banking sector. You will work with Trino, Kafka, Iceberg and Dataiku to ensure data quality and system integrity.

Responsibilities include building scalable test frameworks, automating CI/CD pipelines, testing streaming pipelines, and performing resilience testing to keep the platform stable and reliable.