Data Engineer – Azure & Databricks

Pauwels Consulting; Brussel; Sur place; EUR 65 000 - 90 000; Plein temps

Résumé du poste

Pauwels Consulting in Brussels seeks a data engineer to join the data management team. The role focuses on analyzing, implementing, and maintaining ETL pipelines in an Azure environment to support data warehousing and data lake initiatives.

You will design high-performance data solutions, ensure robust testing, and optimize pipelines during production. Collaboration with teams and fluent Dutch/French with English are valued assets.

Qualifications

• 3+ years of experience with Azure cloud data platforms including Data Lake Gen2 and Synapse.
• 3+ years of experience with Databricks and Python programming for data engineering.
• 3+ years of experience in Azure Data Factory and Data Flow.
• 3+ years of experience in DevOps practices, including Infrastructure as Code and CI/CD.
• 3+ years of experience in Power BI and semantic data modeling.
• Knowledge of metadata management, lineage, and data quality.
• Experience with database modeling using Erwin or Visio.
• Ability to translate complex business requirements into technical specs.

Responsabilités

• Analyze, implement, and maintain ETL processes using Azure Data Factory and PySpark in Databricks.
• Design and develop high-performance data solutions based on functional requirements.
• Prepare and execute comprehensive testing activities to ensure the reliability of data pipelines.
• Perform bug fixing and optimization of pipelines during daily production runs.
• Gather and translate business requirements into detailed functional and technical specifications.
• Manage database modeling tasks using Erwin or Visio.

Connaissances

ETL design

Data modeling

Metadata mgmt

Data quality

CI/CD

DevOps

Python programming

BI reporting

Outils

Azure Data Factory

Databricks

Python

Power BI

Erwin

Visio

Azure Synapse

Data Lake Gen2

Description du poste

Our client, a leading organization in the insurance sector, is seeking a professional to join their data management team to enhance the capacity of their data platform. The role focuses on the analysis, implementation, and maintenance of ETL processes within an Azure environment to support data warehousing and data lake initiatives.

Responsibilities

• Analyze, implement, and maintain ETL processes using Azure Data Factory and PySpark in Databricks.
• Design and develop high-performance data solutions based on specific functional requirements.
• Prepare and execute comprehensive testing activities to ensure the reliability of data pipelines.
• Perform bug fixing and optimization of pipelines during daily production runs.
• Gather and translate business requirements into detailed functional and technical specifications.
• Manage database modeling tasks using specialized tools such as Erwin or Visio.

Requirements

• 3+ years of experience with Azure cloud data platforms including Data Lake Gen2 and Synapse.
• 3+ years of experience with Databricks and Python programming for data engineering.
• 3+ years of experience in Azure Data Factory and Data Flow.
• 3+ years of experience in DevOps practices, including Infrastructure as Code and CI/CD.
• 3+ years of experience in Power BI and semantic data modeling.
• You bring knowledge of data management principles such as metadata management, lineage, and data quality.
• You possess experience in database modeling using Erwin or Visio.
• You have the ability to handle analytics use cases and translate complex business requirements.
• You're proactive, solution-oriented, and able to work both independently and in a collaborative team environment.
• You are fluent in Dutch or French with basic knowledge of the other, and have a good command of English.

Nice to Haves

• Experience within the insurance or financial domain.
• Experience on projects involving different technology stacks and integration aspects.
• Professional and client-friendly approach.

Offer

• Start date: ASAP.
• Duration: 12 months.
• Work regime: Full-time.
• Location: Brussels.
• Contract: open to both permanent employees and freelancers.