Senior Data Engineer

In4Matic; Brussel; Sur place; EUR 70 000 - 100 000; Plein temps

Résumé du poste

In4Matic is seeking an experienced Data Engineer to join the team and contribute to the development of a secure, on-premise data platform in a highly regulated environment. You will collaborate with architecture and BI specialists to build robust data solutions emphasizing security, governance, traceability and data sovereignty.

The role focuses on designing ingestion pipelines, implementing SCD patterns, Delta Lake and end-to-end data lineage, and exposing curated data to BI dashboards in a

Qualifications

• Strong Python and SQL expertise.
• Experience with dbt-core and data modelling.
• Familiarity with data governance and metadata platforms.

Responsabilités

• Design and deploy data ingestion pipelines from heterogeneous sources.
• Implement incremental and streaming ingestion with end-to-end traceability.
• Develop transformation models following a Medallion Architecture (Bronze, Silver, Gold).
• Implement data historization patterns (SCD Type 2, upserts, append-only).
• Develop and maintain Delta Lake tables and exports with ACID, time travel, and optimization.
• Build and maintain orchestration assets; monitor runs; automate alerts.
• Manage metadata; support data lineage and governance documentation.
• Expose Gold-layer data to BI dashboards in collaboration with reporting teams.

Connaissances

Python

Advanced SQL

dbt-core

Delta Lake

DuckDB

MSSQL

PostgreSQL

Airflow

Linux/OpenShift

PowerShell

Data governance

DataHub/OpenMetadata

Outils

dbt-core

DuckDB

Delta Lake

MSSQL

PostgreSQL

Dagster

Airflow

Linux/OpenShift

PowerShell

DataHub

OpenMetadata

Description du poste

We’re looking for an experienced Data Engineer to join our client’s team and contribute to the development of a secure, on-premise data platform in a highly regulated environment. You’ll work as part of a multidisciplinary data team alongside architecture and BI specialists, helping build robust data solutions where security, governance, traceability and data sovereignty are key priorities.

Role & Responsibilities

• Design and deploy reliable data ingestion pipelines from heterogeneous sources, including files, APIs and relational databases.
• Implement incremental and streaming ingestion, ensuring reliability, idempotency and end-to-end traceability.
• Develop transformation models following a Medallion Architecture (Bronze, Silver and Gold).
• Implement data historization patterns such as SCD Type 2, upserts and append-only models.
• Develop and maintain Delta Lake tables and exports, leveraging ACID transactions, time travel and optimization capabilities.
• Build and maintain orchestration assets, monitor pipeline runs, handle failures and automate alerts.
• Manage technical and business metadata, contribute to end-to-end data lineage and support the production of governance documentation.
• Develop endpoints to expose curated Gold-layer data to BI dashboards and collaborate closely with reporting teams.

Technical Profile

The ideal candidate has strong hands‑on experience with Python and advanced SQL, as well as proven expertise with dbt-core, including models, snapshots, macros, tests and profiles. You should be comfortable working with DuckDB, Delta Lake, MSSQL and PostgreSQL, including advanced SQL and CDC concepts, and have experience with an orchestration framework such as Dagster, Airflow or an equivalent solution. Experience with Linux/OpenShift and PowerShell is expected, together with familiarity with data governance and metadata platforms such as DataHub or OpenMetadata. A solid understanding of data engineering, data architecture, Data Lakes, DWH and Lakehouse/Medallion concepts, combined with BI development experience, is highly valued.

Nice to Have

• Experience with Kafka and Debezium for real‑time data capture.
• Knowledge of Apache Spark, including Spark SQL, Thrift Server and Beeline.
• Experience with Power BI Report Server (PBIRS) or Apache Superset.
• Understanding of security requirements in restricted‑access environments.
• Practical knowledge of data governance, metadata management and end‑to‑end lineage.
• Previous experience working in complex, highly governed or security‑sensitive environments.

Contactperson & Reference

• Reference #: INW27553
• Pieter Messely
• [e-mail]