Azure Data Engineer

Community Consulting; Brussel Hoofdstad; Híbrido; EUR 70.000 - 95.000; Tempo integral

Resumo da vaga

Community Consulting in Brussels is seeking an experienced Azure Data Engineer to advance its Azure Databricks platform, governance with Unity Catalog, and cost optimization initiatives. You will design and operate data pipelines with Azure Data Factory, Delta Lake, and Lakehouse architectures, working with analysts and scientists to deliver scalable data solutions.

Fluency in French or Dutch and English is required; a CS/data-related degree and 5+ years of relevant experience are expected.

Qualificações

• Minimum 5 years of experience in a similar Azure/Data Engineer role.
• Practical experience with Databricks Unity Catalog including access management and governance.
• Strong Python and SQL programming with Spark/PySpark.
• Experience with Azure Data Factory, Delta Lake and Lakehouse architectures.
• Excellent communication in French or Dutch and English.

Responsabilidades

• Design, develop and maintain data pipelines using Azure Data Factory, Azure Databricks and other Azure services.
• Implement modern data architectures including Data Lake, Delta Lake and Lakehouse.
• Develop and industrialize new business data use cases.
• Implement and manage Databricks Unity Catalog, including catalogs, schemas, permissions, data sharing and governance.
• Support the implementation of a secure and governed self-service data strategy.
• Monitor and optimize Azure and Databricks performance, resource consumption and costs.
• Automate data ingestion, transformation and deployment processes.
• Collaborate with Data Analysts, Data Scientists and business stakeholders.
• Document data flows, architecture and technical best practices.
• Maintain and continuously improve the Azure Data Platform to ensure performance, reliability, security and scalability.

Conhecimentos

Databricks Unity Catalog

Azure Databricks

Python

SQL

Spark / PySpark

Data pipelines

Cost optimization

CI/CD for data

REST API

AMQP

Formação

Bachelor's degree in Computer Science, Data Engineering or related field

Ferramentas

Azure Data Factory

Delta Lake

Descrição da vaga

As part of its 2027 roadmap, a major Brussels-based public transport organization is continuing to modernize and optimize its Azure Data Platform following the migration of non-SAP data to Microsoft Azure.

The organization aims to develop new reporting, analytics and Data Science use cases, strengthen data governance through Databricks Unity Catalog, expand secure self-service data capabilities, optimize Azure and Databricks consumption and costs, and continuously improve platform performance, scalability and reliability.

An experienced Azure Data Engineer is needed to support the Data & Analytics team in delivering reliable, scalable and secure data solutions while evolving the Azure Databricks architecture.

Responsibilities

• Design, develop and maintain data pipelines using Azure Data Factory, Azure Databricks and other Azure services.
• Implement modern data architectures including Data Lake, Delta Lake and Lakehouse.
• Develop and industrialize new business data use cases.
• Implement and manage Databricks Unity Catalog, including catalogs, schemas, permissions, data sharing and governance.
• Support the implementation of a secure and governed self-service data strategy.
• Monitor and optimize Azure and Databricks performance, resource consumption and costs.
• Automate data ingestion, transformation and deployment processes.
• Collaborate with Data Analysts, Data Scientists and business stakeholders.
• Document data flows, architecture and technical best practices.
• Maintain and continuously improve the Azure Data Platform to ensure performance, reliability, security and scalability.

Must have

• Minimum 5 years of experience in a similar role.
• Degree in Computer Science, Data Engineering or a related field.
• Proven practical experience with Databricks Unity Catalog, including access management, security and data governance.
• Strong understanding of Azure Databricks architecture, Lakehouse design, scalability, security and best practices.
• Experience with Apache Spark, PySpark and Delta Lake.
• Strong programming and querying skills in Python and SQL.
• Experience with performance monitoring, tuning and cost optimization across Azure and Databricks.
• Knowledge of CI/CD principles applied to Data Engineering.
• Knowledge of application integration patterns, including REST API and AMQP.
• Strong analytical thinking, autonomy, problem-solving abilities and team spirit.
• Strong communication skills and ability to explain technical concepts clearly.
• Excellent command of either French or Dutch.
• Good command of the other national language (French or Dutch).
• Good command of English.

Should have

• Microsoft Azure or Databricks certifications.
• Experience with Power BI administration and governance.
• Experience with real-time data streaming.
• Experience with Terraform or Infrastructure as Code.
• Familiarity with Azure Monitor and Log Analytics.
• Basic knowledge of SAP S/4HANA or BW/4HANA as data sources.
• Experience working in Agile/Scrum environments.

Who we are

Community Consulting goes beyond traditional consulting; it’s all about fostering connections in an atmosphere of trust and confidence.
Transparency & Honesty : We say things as they are. Clear communication for seamless collaboration.
#COMMUNITEAM : Work independently, but never alone. Collective intelligence drives us further, faster.
Total Commitment : Always present, always engaged. We find solutions and make sure everyone moves forward together.
Guaranteed Efficiency : No fluff, just results. We act fast, keep our promises, and deliver top quality.
This is our DNA. This is how we make a difference.