Senior Data Engineer – Azure

Aurora Engineering AB; Göteborgs kommun; On-site; SEK 900,000 - 1,100,000; Full time

Job summary

Aurora Engineering AB in Gothenburg, Sweden is seeking an experienced Senior Data Engineer to design, build, and operate scalable data pipelines in the Microsoft Azure ecosystem, with a focus on production-ready solutions.

You will work across batch and streaming data, ETL/ELT processes, API and system integrations, and incorporate AI/ML models into production data flows, leveraging Azure Data Factory, Databricks, Delta Lake, and CI/CD via Azure DevOps.

Qualifications

• 5+ years of relevant Data Engineering experience.
• Hands-on expertise with Microsoft Azure ecosystem.
• Experience with system/API integration, streaming architectures, CI/CD, and AI model integration.

Responsibilities

• Design, build, and maintain scalable data pipelines in Microsoft Azure
• Develop ETL/ELT solutions using Azure Data Factory, Azure Databricks, and Blob Storage
• Transform and process large structured and unstructured datasets using Python and PySpark
• Build and maintain both batch and streaming data pipelines
• Work with event‑driven architectures using technologies such as Azure Service Bus, Kafka, or equivalent
• Design and optimize data models in Azure Databricks and Delta Lake
• Improve pipeline performance through partitioning, optimization, and performance tuning
• Implement and maintain CI/CD pipelines using Azure DevOps
• Ensure data quality, monitoring, reliability, and stability in production environments
• Develop and maintain API and enterprise system integrations
• Integrate AI/ML models with applications, APIs, data pipelines, and production systems

Skills

Python

PySpark

Azure

CI/CD

Data pipelines

Problem solving

Tools

Azure Data Factory

Azure Databricks

Delta Lake

Kafka

Azure Service Bus

Azure DevOps

API integration

Job description

We are looking for an experienced Senior Data Engineer with strong hands‑on expertise in building scalable, reliable, and production‑ready data solutions within the Microsoft Azure ecosystem.

The ideal candidate combines strong data engineering capabilities with experience in system/API integration, streaming architectures, CI/CD, and AI model integration.

Key Responsibilities

• Design, build, and maintain scalable data pipelines in Microsoft Azure
• Develop ETL/ELT solutions using Azure Data Factory, Azure Databricks, and Blob Storage
• Transform and process large structured and unstructured datasets using Python and PySpark
• Build and maintain both batch and streaming data pipelines
• Work with event‑driven architectures using technologies such as Azure Service Bus, Kafka, or equivalent
• Design and optimize data models in Azure Databricks and Delta Lake
• Improve pipeline performance through partitioning, optimization, and performance tuning
• Implement and maintain CI/CD pipelines using Azure DevOps
• Ensure data quality, monitoring, reliability, and stability in production environments
• Develop and maintain API and enterprise system integrations
• Integrate AI/ML models with applications, APIs, data pipelines, and production systems

Required Qualifications

• 5+ years of relevant Data Engineering experience
• Strong hands‑on experience with Microsoft Azure
• Advanced proficiency in Python and PySpark
• Strong experience with Azure Data Factory and Azure Databricks
• Experience with Delta Lake, data modeling, partitioning, and performance optimization
• Experience with both streaming and batch processing
• Knowledge of Kafka, Azure Service Bus, or similar event‑driven technologies
• Strong experience with Azure DevOps and CI/CD
• Experience with API integration and system integration
• Understanding of production‑level data quality, monitoring, and reliability
• Experience connecting AI/ML models to production systems
• Strong analytical, troubleshooting, and problem‑solving skills

Important: Candidates must currently be located in Gothenburg, Sweden.