Senior Big Data System Engineer (Platform Engineer)

EPAM Systems; Schweiz; Vor Ort; CHF 100.000 - 130.000; Vollzeit

Zusammenfassung

EPAM Systems is looking for a Senior Big Data System Engineer located in Switzerland. This role involves maintaining and enhancing distributed systems and data pipelines within a dynamic banking environment.

The ideal candidate will have over 5 years of experience in building fault-tolerant systems, with expert knowledge in Kafka, Kubernetes, and Python. This position offers a hybrid work model combining remote work and on-site presence at our client’s office in Zürich.

Qualifikationen

• 5+ years of experience in large-scale distributed systems.
• Expert knowledge of Kafka, Kubernetes, and Spark.
• Proficiency in Python or Java programming.

Aufgaben

• Operate and maintain core Global Data Platform components.
• Manage data and analytics applications.
• Automate infrastructure configurations and CI/CD pipelines.

Kenntnisse

Distributed data technologies

Kafka

Kubernetes

Python

Java

Linux/Unix scripting

CI/CD pipelines

Tools

Docker

Apache stack components

Dataiku

Collibra

Jobbeschreibung

Are you an experienced Big Data professional eager to tackle complex challenges in a dynamic banking environment? EPAM is seeking a dedicated Senior Big Data System Engineer (Platform Engineer) to maintain, optimize and enhance distributed systems and data pipelines for one of our strategic clients in the financial sector. In this role, you will operate global data platform components, manage applications and integrate cutting-edge tools to deliver scalable and resilient solutions. We offer a hybrid work model with a mix of remote and on-site work at our client’s office in Zürich.

Responsibilities

• Operate and maintain core Global Data Platform components, including VM Servers, Kubernetes and Kafka, to ensure smooth system performance and functionality.
• Manage data and analytics applications, such as Apache stack components, Dataiku, Collibra and other essential tools.
• Automate infrastructure configurations, security components and CI/CD pipelines to drive efficiency and eliminate manual intervention in data pipelines.
• Develop robust solutions to enhance platform resiliency, implement health checks, monitoring, alerting and self-recovery mechanisms for data operations.
• Focus on improving data pipeline quality by addressing accuracy, timeliness and recency in ELT/ETL execution.
• Embed Agile and DevSecOps best practices into delivery processes, ensuring iterative development and deployment of integrated solutions.
• Collaborate with stakeholders like enterprise security, digital engineering and cloud operations to align on effective solution architectures.
• Keep track of and evaluate emerging technologies across the Big Data ecosystem to deliver innovative features and capabilities.

Requirements

• Demonstrated 5+ years of experience in building or designing fault-tolerant, large-scale distributed systems, showcasing an ability to manage complex infrastructure and operations.
• Mastery of distributed data technologies such as data lakes, delta lakes, data meshes, data lakehouses and real-time streaming platforms.
• Expert knowledge of tools like Kafka, Kubernetes and Spark, as well as formats like S3/Parquet for scalable data solutions.
• Proficiency in Python and Java programming, or alternatives such as Scala/R, coupled with adeptness in Linux/Unix scripting.
• Experience in managing Docker (Harbor), VM setup/scaling, Kubernetes pod management and CI/CD pipelines.
• Knowledge of configuration management tools like Jinja templates, puppet scripts and best practices in firewall rules setup.
• Strong understanding of DevOps practices for scalable deployment and automation strategies.
• Fluency in English is essential, German language skills are considered an asset for collaboration.
• Familiarity with financial services and their unique data and regulatory challenges is an advantage.