Azure Data Engineer — Scalable Pipelines, AI & Streaming

Aurora Engineering AB; Göteborgs kommun; On-site; SEK 900,000 - 1,100,000; Full time

Job summary

Aurora Engineering AB in Gothenburg, Sweden is seeking an experienced Senior Data Engineer to design, build, and operate scalable data pipelines in the Microsoft Azure ecosystem, with a focus on production-ready solutions.

You will work across batch and streaming data, ETL/ELT processes, API and system integrations, and incorporate AI/ML models into production data flows, leveraging Azure Data Factory, Databricks, Delta Lake, and CI/CD via Azure DevOps.

Qualifications

• 5+ years of relevant Data Engineering experience.
• Hands-on expertise with Microsoft Azure ecosystem.
• Experience with system/API integration, streaming architectures, CI/CD, and AI model integration.

Responsibilities

• Design, build, and maintain scalable data pipelines in Microsoft Azure
• Develop ETL/ELT solutions using Azure Data Factory, Azure Databricks, and Blob Storage
• Transform and process large structured and unstructured datasets using Python and PySpark
• Build and maintain both batch and streaming data pipelines
• Work with event‑driven architectures using technologies such as Azure Service Bus, Kafka, or equivalent
• Design and optimize data models in Azure Databricks and Delta Lake
• Improve pipeline performance through partitioning, optimization, and performance tuning
• Implement and maintain CI/CD pipelines using Azure DevOps
• Ensure data quality, monitoring, reliability, and stability in production environments
• Develop and maintain API and enterprise system integrations
• Integrate AI/ML models with applications, APIs, data pipelines, and production systems

Skills

Python

PySpark

Azure

CI/CD

Data pipelines

Problem solving

Tools

Azure Data Factory

Azure Databricks

Delta Lake

Kafka

Azure Service Bus

Azure DevOps

API integration

Job description

Aurora Engineering AB in Gothenburg, Sweden is seeking an experienced Senior Data Engineer to design, build, and operate scalable data pipelines in the Microsoft Azure ecosystem, with a focus on production-ready solutions.

You will work across batch and streaming data, ETL/ELT processes, API and system integrations, and incorporate AI/ML models into production data flows, leveraging Azure Data Factory, Databricks, Delta Lake, and CI/CD via Azure DevOps.