Medior Data Engineer (Databricks / Azure, Treasury Data Warehouse)

Koderia; Wien; Vor Ort; EUR 60.000 - 80.000; Vollzeit

Zusammenfassung

Koderia is seeking a Medior Data Engineer in Vienna to support the modernization of a Treasury Data Warehouse. The role involves migrating ETL pipelines from MS SQL Server to Databricks on Azure, and requires strong skills in Python, SQL, and Spark.

The ideal candidate will have experience in designing workflows, ensuring data governance, and performing validation checks. A hybrid work model allows for flexibility with 2 days on-site per week, and the position is a 6-month contract with potential extension.

Qualifikationen

• Strong hands-on experience with Python and SQL.
• Experience with Apache Spark and Databricks on Azure.
• Knowledge of GDPR and data governance topics.

Aufgaben

• Design and migrate ETL pipelines from MS SQL Server to Databricks.
• Convert T-SQL logic into Spark jobs and optimize performance.
• Implement data governance policies and security.

Kenntnisse

Python

SQL

Transact-SQL

Apache Spark

Databricks

ETL

SSIS

Data governance

Tools

Databricks

Tableau

Power BI

Azure Data Factory

Jobbeschreibung

This project is already taken or is out of date.

Medior Data Engineer (Databricks / Azure, Treasury Data Warehouse)

Location Vienna

Experience Medior

We are looking for Medior Data Engineer to support a modernization of a Treasury Data Warehouse (TWH) environment, including migration from an MS-SQL-based operational DWH with SSAS/ROLAP cubes to Databricks/DBX on Azure. The engagement focuses on building data ingestion and ETL pipelines, converting existing logic to Spark/Databricks jobs, and supporting governance, validation, and business continuity for reporting in a hybrid setup across Vienna and Bratislava.

Key Responsibilities

• Design, develop, and migrate data ingestion and ETL pipelines from MS SQL Server / SSIS to Databricks (Spark) on Azure.
• Convert existing T‑SQL / Transact‑SQL logic into Spark/Databricks jobs (e.g., Python/SQL-based implementations) and optimize performance.
• Implement and maintain workflow orchestration / workflow management (e.g., Databricks Workflows and related orchestration approaches; ADF where applicable).
• Support definition and delivery of PoC/MVP scope, including functional and non‑functional requirements, acceptance criteria, and validation of migration/design options.
• Work with target architecture patterns such as Bronze/Silver/Gold layers and align engineering work with the target data model and TOM.
• Implement reconciliation checks and validation to compare SQL Server outputs vs Databricks outputs; contribute to data quality KPIs.
• Contribute to data governance and security implementation: data access policies, encryption, cataloging and lineage (e.g., Unity Catalog and lineage/validation tooling).
• Support business continuity for reporting by redesigning/adapting dependent assets such as reports, dashboards, cubes, and semantic models (e.g., Tableau, XLCubed, Power BI).
• Collaborate with stakeholders across delivery leadership, data engineering, governance, and business/domain analysis teams in Vienna and Bratislava; contribute to stakeholder communication and coordination.

Requirements

• Seniority: Medior.
• Strong hands‑on experience with Python and SQL, including Transact‑SQL.
• Experience with Apache Spark and Databricks/DBX (target platform on Azure preferred).
• Experience with Extract Transform Load (ETL) and migration/rewriting of pipelines.
• Experience with SQL Server Integration Services (SSIS); familiarity with SQL Server and DWH environments (including SSAS/ROLAP cubes context).
• Experience with workflow management / orchestration (e.g., Databricks workflows; ADF mentioned as relevant).
• Knowledge of governance and compliance topics relevant to data platforms, including GDPR and frameworks such as DAMA.
• Familiarity with cataloging/lineage and access control approaches (e.g., Unity Catalog).
• Technologies mentioned as relevant: ANTLR, data lineage/validation tools, Tableau, XLCubed, Power BI.
• Preferred: Treasury business know-how; prior experience with Erste Group (not mandatory, but preferred in the broader program context).

Other

• Work mode: Hybrid with 2 days on-site per week.
• Contract length: 6 months (possible extension: yes).

Platform/Tool Tableau Azure Data Factory databricks Unity catalog Azure Power BI SQL Server Analysis Services (SSAS) XLCubed SQL Server Integration Services (SSIS) Databricks Workflows