ETL & Data Pipeline Engineer - Azure & Kafka

Alior Bank SA; Województwo pomorskie; Hybride; PLN 180 000 - 280 000; Plein temps

Avantages offerts par ce poste

Kafeteria

Prywatna opieka medyczna z pakietem St

Karta Multisport

Ubezpieczenie

Konkursy wewnętrzne

Programy zniżkowe i lojalnościowe

Dzień na U – 16 godzin wolontariatu

Alior Uniwersytet

Résumé du poste

Alior Bank SA poszukuje specjalisty ds. hurtowni danych i ETL. Będziesz projektować i utrzymywać widoki oraz struktury danych, wspierać procesy integracyjne oraz zasileń danych w hurtowni Teradata.

Wymagane doświadczenie z Azure Data Factory, Apache Kafka, Spark i Git, a także praktyczna znajomość DataOps i CI/CD. Praca w zespole Scrum w modelu hybrydowym, z atrakcyjnym pakietem benefitów.

Qualifications

• Minimum 3 years of documented experience in a similar role.
• Practical experience with ETL solutions.
• Good or very good SQL skills and experience with databases.
• Experience designing and optimizing data structures and query performance.
• Knowledge of Microsoft Azure services related to storage, integration and data processing.
• Experience with Azure Data Factory (ADF).
• Experience processing large data sets, e.g. with Apache Spark or Azure Databricks.
• Knowledge of Apache Kafka and event-driven architecture.

Responsabilités

• Design, build and maintain data views and structures used by integration processes.
• Design, build and maintain ETL processes, optimize performance and data flows.
• Develop and maintain batch data loads: Data Warehouse (Teradata) → Azure Data Factory → Adobe solutions.
• Develop and maintain Real-Time data loads and integrations (Kafka → Azure Event Hubs → Adobe Accelerato r).
• Configure, operate and diagnose Kafka-based integrations including topics, producers, consumers, partitions and data streams.
• Collaborate with source teams to identify and run new data sources and streams.
• Monitor pipelines and data flows, analyze errors, delays and performance issues.
• Develop and implement DataOps standards and ensure data quality, completeness, consistency and security.
• Support performance tests for integrations and Real-Time data, diagnose end-to-end issues from source to target.
• Develop automation for CI/CD for data and ETL processes.
• Create and maintain process and pipeline documentation.
• Work in an agile Scrum team.

Connaissances

SQL

Azure Data Factory

Azure Databricks

Apache Spark

Apache Kafka

ETL design

Teradata

Git

DevOps / CI/CD

Generative AI

Outils

Teradata

Microsoft Azure

Azure Event Hubs

Description du poste

Alior Bank SA poszukuje specjalisty ds. hurtowni danych i ETL. Będziesz projektować i utrzymywać widoki oraz struktury danych, wspierać procesy integracyjne oraz zasileń danych w hurtowni Teradata.

Wymagane doświadczenie z Azure Data Factory, Apache Kafka, Spark i Git, a także praktyczna znajomość DataOps i CI/CD. Praca w zespole Scrum w modelu hybrydowym, z atrakcyjnym pakietem benefitów.