Azure Data Engineer - Databricks, Unity Catalog & Lakehouse

Pauwels Consulting; Brussel; Hybrid; EUR 70,000 - 110,000; Full time

Job summary

Pauwels Consulting is seeking an experienced Azure Data Engineer to strengthen the data and analytics team in Brussels. The role focuses on developing, optimizing, and governing a modern data platform to support advanced business use cases and data science initiatives.

You will design and implement scalable data pipelines with Azure Data Factory and Databricks, establish Lakehouse patterns, and manage Unity Catalog for governance.

Qualifications

• 5+ years of experience in data engineering with a focus on Azure cloud environments.
• Deep expertise in Azure Databricks and Unity Catalog, including Lakehouse design and security.
• Advanced skills in Spark, PySpark, and Delta Lake for large-scale data processing.
• Strong proficiency in SQL and Python for data manipulation and platform automation.
• Experience with Azure Data Factory, Azure Storage, Azure SQL, and Key Vault.
• Knowledge of CI/CD principles and Azure DevOps applied to data engineering.
• Familiarity with REST API and AMQP.
• Fluent in French or Dutch with professional proficiency in English.

Responsibilities

• Design and develop scalable data pipelines using Azure Data Factory and Databricks.
• Implement modern data architectures including Data Lake, Delta Lake, and Lakehouse patterns.
• Manage Databricks Unity Catalog for data sharing, permissions, and governance.
• Automate data ingestion, transformation, and deployment processes to enhance platform efficiency.
• Monitor and optimize resource consumption and performance within Azure and Databricks environments.
• Collaborate with data scientists and business stakeholders to industrialize new analytics use cases.

Skills

Azure cloud

Databricks

Spark

PySpark

SQL

Python

CI/CD

Azure DevOps

REST API

AMQP

communication

analytical mindset

Tools

Azure Data Factory

Azure Storage

Azure SQL

Key Vault

Delta Lake

Lakehouse

Unity Catalog

Terraform

Power BI

Job description

Pauwels Consulting is seeking an experienced Azure Data Engineer to strengthen the data and analytics team in Brussels. The role focuses on developing, optimizing, and governing a modern data platform to support advanced business use cases and data science initiatives.

You will design and implement scalable data pipelines with Azure Data Factory and Databricks, establish Lakehouse patterns, and manage Unity Catalog for governance.