Senior Azure Data Engineer — Databricks Lakehouse

Software Mind; Kraków; On-site; PLN 180,000 - 240,000; Full time

Benefits offered by this job

Flexible remote work

Private healthcare

Multisport card

Language classes

Training opportunities

Job summary

Software Mind Poland is seeking a Senior Data Engineer with Databricks PySpark expertise to join our Data Engineering Team. You will build reusable data pipelines on Databricks Lakehouse, develop comprehensive data models, and implement incremental loads in Azure‑based platforms.

You will collaborate with infrastructure teams to set up cloud resources, optimize performance with techniques like zOrder and ACID transactions, and contribute to event‑driven architectures across large datasets.

Qualifications

• Databricks Lakehouse and PySpark development experience.
• Extensive data modeling with fact/dimension tables and SCD Type 2.
• Advanced Python and PySpark for ETL/ELT tasks.
• Knowledge of Azure data platform and SQL for complex transformations.
• Experience with incremental data loads and idempotent patterns.
• Familiarity with CI/CD and agile development practices.

Responsibilities

• Develop reusable, metadata‑driven data pipelines using Databricks Lakehouse and PySpark.
• Design and implement comprehensive data models.
• Build robust ETL/ELT solutions with advanced features like SCD Type 2.
• Implement incremental loads with idempotency and overlap joins.
• Automate and optimize data platform processes for performance and reliability.
• Create event‑driven data integrations and collaborate with infra teams to provision cloud resources.

Skills

Databricks Lakehouse

PySpark

Python

SQL

Data modeling

SCD Type 2

Incremental loads

Event‑driven architecture

Agile / Scrum

CI/CD

Azure data platform

Tools

Terraform

Bicep

Azure Data Factory

Azure Function App

Azure Container Apps

Kubernetes

Job description

Software Mind Poland is seeking a Senior Data Engineer with Databricks PySpark expertise to join our Data Engineering Team. You will build reusable data pipelines on Databricks Lakehouse, develop comprehensive data models, and implement incremental loads in Azure‑based platforms.

You will collaborate with infrastructure teams to set up cloud resources, optimize performance with techniques like zOrder and ACID transactions, and contribute to event‑driven architectures across large datasets.