Senior Data Engineer - Azure Databricks Data Platform

emagine; København; On-site; DKK 850,000 - 1,100,000; Full time

Job summary

emagine seeks a Senior Data Engineer to build and operate a data platform for regulated labeling and eLabel data in an Azure Databricks Lakehouse. You will develop scalable pipelines with a strong focus on quality, governance and traceability, handling Bronze→Silver→Gold architecture and metadata lineage.

Based in Greater Copenhagen, the role involves Azure storage, Databricks workflows, and CI/CD for notebooks, with emphasis on reliable data processing in a regulated environment. 37 hours/week.

Qualifications

• Hands-on Azure Databricks engineering experience.
• Strong data governance and quality focus.
• Experience in regulated environments.

Responsibilities

• Develop and maintain data pipelines using Azure Databricks, Python/PySpark, Spark SQL and Delta Lake.
• Build Bronze → Silver → Gold medallion architectures.
• Ingest data from APIs, JSON/files, Azure Storage and enterprise systems.
• Implement data quality, validation, reconciliation and automated testing.
• Handle schema enforcement/evolution, metadata and data lineage.
• Optimize pipeline performance, partitioning and scalability.
• Implement CI/CD, monitoring, error handling and reprocessing.
• Ensure traceability, auditability and reproducibility of regulated data pipelines.

Skills

Azure Databricks

Python/PySpark

Spark SQL

Delta Lake

Bronze → Silver → Gold

ADLS Gen2/Azure Storage

Databricks Workflows/Jobs

Unity Catalog & governance

API data ingestion

CI/CD for Databricks

Metadata & data lineage

Delta Live Tables

Lakeflow (advantage)

Tools

Delta Live Tables

Lakeflow

Job description

emagine seeks a Senior Data Engineer to build and operate a data platform for regulated labeling and eLabel data in an Azure Databricks Lakehouse. You will develop scalable pipelines with a strong focus on quality, governance and traceability, handling Bronze→Silver→Gold architecture and metadata lineage.

Based in Greater Copenhagen, the role involves Azure storage, Databricks workflows, and CI/CD for notebooks, with emphasis on reliable data processing in a regulated environment. 37 hours/week.