Azure Data Engineer: Databricks & Lakehouse Architect

Community Consulting; Brussel Hoofdstad; Hybrid; EUR 70,000 - 95,000; Full time

Job summary

Community Consulting in Brussels is seeking an experienced Azure Data Engineer to advance its Azure Databricks platform, governance with Unity Catalog, and cost optimization initiatives. You will design and operate data pipelines with Azure Data Factory, Delta Lake, and Lakehouse architectures, working with analysts and scientists to deliver scalable data solutions.

Fluency in French or Dutch and English is required; a CS/data-related degree and 5+ years of relevant experience are expected.

Qualifications

• Minimum 5 years of experience in a similar Azure/Data Engineer role.
• Practical experience with Databricks Unity Catalog including access management and governance.
• Strong Python and SQL programming with Spark/PySpark.
• Experience with Azure Data Factory, Delta Lake and Lakehouse architectures.
• Excellent communication in French or Dutch and English.

Responsibilities

• Design, develop and maintain data pipelines using Azure Data Factory, Azure Databricks and other Azure services.
• Implement modern data architectures including Data Lake, Delta Lake and Lakehouse.
• Develop and industrialize new business data use cases.
• Implement and manage Databricks Unity Catalog, including catalogs, schemas, permissions, data sharing and governance.
• Support the implementation of a secure and governed self-service data strategy.
• Monitor and optimize Azure and Databricks performance, resource consumption and costs.
• Automate data ingestion, transformation and deployment processes.
• Collaborate with Data Analysts, Data Scientists and business stakeholders.
• Document data flows, architecture and technical best practices.
• Maintain and continuously improve the Azure Data Platform to ensure performance, reliability, security and scalability.

Skills

Databricks Unity Catalog

Azure Databricks

Python

SQL

Spark / PySpark

Data pipelines

Cost optimization

CI/CD for data

REST API

AMQP

Education

Bachelor's degree in Computer Science, Data Engineering or related field

Tools

Azure Data Factory

Delta Lake

Job description

Community Consulting in Brussels is seeking an experienced Azure Data Engineer to advance its Azure Databricks platform, governance with Unity Catalog, and cost optimization initiatives. You will design and operate data pipelines with Azure Data Factory, Delta Lake, and Lakehouse architectures, working with analysts and scientists to deliver scalable data solutions.

Fluency in French or Dutch and English is required; a CS/data-related degree and 5+ years of relevant experience are expected.