Senior Data Engineer / IT Developer – Azure Databricks

Aurora Engineering AB; Stockholms kommun; On-site; SEK 900,000 - 1,400,000; Full time

Job summary

Aurora Engineering AB in Stockholm is seeking an experienced Senior Data Engineer / IT Developer for a prominent assignment with a large organization. You will design and implement dimensional and semantic data models, and build Bronze/Silver/Gold layers using medallion architecture.

You'll develop scalable pipelines using Azure Databricks, Delta Lake, Lakeflow, and Unity Catalog, while ensuring data quality, governance, and GDPR compliance.

Qualifications

• Senior-level professional experience in data engineering or data platform development.
• Hands-on experience with Azure Databricks, Delta Lake, Lakeflow, and Unity Catalog.
• Advanced proficiency in SQL, Python, and PySpark.
• Experience developing and orchestrating large-scale pipelines using Azure Data Factory.
• Strong knowledge of dimensional and semantic data modeling, including star schemas, SCD, role-playing dimensions, and calculation groups.
• Experience integrating data from Workday, Peakon, or similar HR/employee/financial systems.
• Experience implementing automated data-quality checks, testing, validation and monitoring.
• Knowledge of GDPR, data protection, access controls, and information governance.

Responsibilities

• Design and develop dimensional and semantic data models.
• Build Bronze, Silver, and Gold layers using Medallion Architecture.
• Develop reusable, scalable datasets with version control.
• Maintain clear data definitions, documentation, and traceability.
• Design, develop, and optimize ETL/ELT pipelines with Azure tools and Python.
• Collaborate with BI teams to optimize Power BI datasets and models.
• Support Agile planning, CI/CD, incident management, and platform improvements.

Skills

Azure Databricks

Delta Lake

Lakeflow

Unity Catalog

SQL

Python

PySpark

Azure Data Factory

Dimensional modeling

SCD

Data governance

GDPR

Stakeholder collaboration

Communication

Job description

We are looking for an experienced Senior Data Engineer / IT Developer to join an exciting assignment with a large organization in Stockholm.

Key Responsibilities

Data Engineering & Data Modeling

• Design and develop dimensional and semantic data models, including star schemas, Slowly Changing Dimensions (SCD), and role-playing dimensions.
• Build and enhance Bronze, Silver, and Gold layers using Medallion Architecture.
• Develop reusable, scalable, and version-controlled datasets.
• Maintain clear data definitions, documentation, and end-to-end traceability.

Data Pipelines & Platform Development

• Design, develop, and optimize ETL/ELT pipelines using Azure Databricks, Azure Data Factory, SQL, and Python/PySpark.
• Work with Delta Lake, Lakeflow, and Unity Catalog.
• Implement incremental data processing and optimize pipeline performance and storage costs.
• Develop reliable data integration solutions across multiple enterprise systems.

Data Quality, Security & Governance

• Implement automated data testing, validation, profiling, and quality monitoring.
• Manage data access and governance using Unity Catalog.
• Apply row-level and column-level security controls.
• Ensure appropriate handling of personal and sensitive information in accordance with GDPR.
• Support data retention policies, documentation, and compliance requirements.

Collaboration & Continuous Improvement

• Translate business requirements into robust technical solutions.
• Collaborate with BI developers to optimize Power BI datasets and semantic models.
• Participate in Agile planning, backlog refinement, and technical discussions.
• Support CI/CD, release management, incident management, and platform improvements.
• Maintain technical documentation and contribute to engineering best practices.

Required Qualifications

• Senior-level professional experience in data engineering or data platform development.
• Strong hands-on experience with Azure Databricks, Delta Lake, Lakeflow, and Unity Catalog.
• Advanced proficiency in SQL, Python, and PySpark.
• Experience developing and orchestrating large-scale pipelines using Azure Data Factory.
• Strong knowledge of dimensional and semantic data modeling, including star schemas, SCD, role-playing dimensions, and calculation groups.
• Experience integrating and modeling data from Workday, Peakon, or equivalent HR, employee experience, or financial systems.
• Experience implementing automated data-quality checks, testing, validation, and monitoring.
• Knowledge of GDPR, data protection, access controls, and information governance.
• Experience working with business stakeholders and translating requirements into scalable technical solutions.
• Strong communication, analytical, and problem-solving skills.