Senior Azure Databricks Data Platform Engineer

OPTIVEUM SPÓŁKA Z OGRANICZONĄ ODPOWIEDZIALNOŚCIĄ; Kraków; Híbrido; PLN 186.000 - 220.000; Jornada completa

Descripción de la vacante

OPTIVEUM SPÓŁKA Z OGRANICZONĄ ODPOWIEDZIALNOŚCIĄ is seeking an Azure Databricks Engineer to join a global investment bank’s team in Kraków. You will design big data solutions using Azure Databricks for batch and real-time analytics.

You will build Spark pipelines feeding BI data marts and enable data science workflows, integrating real-time events from Kafka and persisting results in ADLS Gen2. The role requires 8–10 years of experience with Azure cloud-native services, fluent English, and a

Formación

• Bachelor's degree or higher in Computer Science, Engineering or Science
• 8-10 years of experience, incl. 5+ years with Azure cloud-native services
• Strong hands-on experience with Azure Databricks, Delta Lake, Python / PySpark and Spark
• Knowledge of Lakehouse, Data Mesh, Data Virtualisation, dimensional modelling and data aggregation techniques
• Experience building distributed data platforms (e.g. Spark, Kafka, Flink, HBase) at large scale
• Hadoop-based implementations, SQL / T-SQL, Unix and Hive
• Azure Data Factory, ADLS Gen2 and Azure SQL Database
• Good understanding of RDBMS, Data Lake, Data Warehouse, Lakehouse and Medallion Architecture
• At least one workflow orchestration tool (Apache Airflow, Luigi, Oozie, AWS Glue or similar)
• Knowledge of Microsoft Fabric and basic understanding of data quality dimensions
• Fluent English
• Polish or EU citizenship, or a permanent residence / work permit valid for at least 12 months

Responsabilidades

• Design and implement big data solutions in Azure Databricks for batch and real-time analytics use cases
• Build Spark pipelines that populate downstream data marts for BI and generate feeds for Data Science
• Integrate real-time services that process business events from Kafka and persist them in ADLS Gen2 using Databricks
• Develop near real-time stream processing to derive features for ML model inference
• Optimise Databricks performance (liquid clustering, vacuum, Z-ordering etc.)
• Work with DevOps teams to drive operationally excellent infrastructure

Conocimientos

Azure Databricks

PySpark

Spark

Kafka

Hadoop

SQL

Azure Data Factory

Azure SQL Database

Data Modeling

Airflow

Educación

Bachelor's degree in CS/Engineering/Science

Herramientas

Delta Lake

ADLS Gen2

Azure Data Factory

Azure SQL Database

Airflow

HDFS

Descripción del empleo

OPTIVEUM SPÓŁKA Z OGRANICZONĄ ODPOWIEDZIALNOŚCIĄ is seeking an Azure Databricks Engineer to join a global investment bank’s team in Kraków. You will design big data solutions using Azure Databricks for batch and real-time analytics.

You will build Spark pipelines feeding BI data marts and enable data science workflows, integrating real-time events from Kafka and persisting results in ADLS Gen2. The role requires 8–10 years of experience with Azure cloud-native services, fluent English, and a