Senior Data Engineer

Rhox; Sint-Gillis; Sur place; EUR 90 000 - 120 000; Plein temps

Résumé du poste

Rhox in Belgium is seeking a Senior Data Engineer to design production-grade Python and SQL pipelines. You will ingest data from files, APIs and relational databases, build Bronze, Silver and Gold lakehouse layers with Delta Lake, and expose governed datasets for reporting tools.

Working with a small team including a Data Architect and BI Developer, you will implement Dagster assets, monitor runs, maintain lineage in DataHub and support Power BI consumption with secure, auditable data products.

Qualifications

• Apply Python, SQL, Pandas and Apache Spark to build production pipelines.
• Demonstrate hands-on dbt-core across models, snapshots, tests.
• Understand lakehouse patterns (Bronze/Silver/Gold) and Delta Lake.
• Use Kafka for event-driven ingestion with MSSQL and PostgreSQL.

Responsabilités

• Design and deliver idempotent data acquisition pipelines using ETL and ELT patterns for files, APIs and relational databases, including Kafka-based streaming flows.
• Build Bronze, Silver and Gold transformations with dbt, applying SCD2, upsert and append-only patterns for accurate data history.
• Maintain Delta Lake tables and exports, preserving ACID consistency and reliable access to governed data products.
• Orchestrate Dagster assets, monitor production runs, handle failures and automate alerts so pipelines remain traceable.
• Publish technical and business metadata to DataHub, maintain end-to-end lineage and expose Gold datasets for Power BI reporting.

Connaissances

Python

SQL

Pandas

Apache Spark

Dagster

dbt-core

Delta Lake

Kafka

MSSQL

PostgreSQL

PowerShell

OpenShift

Linux

DataHub

Data governance

Outils

Dagster

dbt-core

Delta Lake

Kafka

PowerShell

OpenShift

Description du poste

A public-sector defence organisation in Belgium is building an on-premise Data Exchange Platform to support secure, sovereign data use across multiple domains. As a Senior Data Engineer in Brussels, you will develop Python and SQL pipelines with Dagster, dbt and Delta Lake, from heterogeneous source ingestion through governed data products and BI exposure. The programme is delivered by a small team in the Belgian defence sector, working with a Data Architect, Solution Architect and BI Developer.

The mission

The platform runs on-premise because the organisation requires strict controls over security, data governance and sovereignty. Its lakehouse design uses Bronze, Silver and Gold layers, Delta Lake tables, ACID transactions and time travel, with dbt and DuckDB supporting analytical transformation. Sources include files, APIs and relational databases, with incremental and streaming ingestion across domains.

You will work in the data engineering stream alongside the Data Architect, Solution Architect and BI Developer. Your scope covers reliable ingestion, transformation, orchestration, metadata management and exposure of the Gold layer to reporting tools. You will implement Dagster assets, monitor runs, maintain end-to-end lineage in DataHub and support BI consumption through Power BI. The role connects production pipeline quality with governance documents, technical decisions and usable datasets for controlled-access users.

Your responsibilities

• Design and deliver idempotent data acquisition pipelines using ETL and ELT patterns for files, APIs and relational databases, including Kafka-based streaming flows.
• Build Bronze, Silver and Gold transformations with dbt, applying SCD2, upsert and append-only patterns for accurate data history.
• Maintain Delta Lake tables and exports, preserving ACID consistency and reliable access to governed data products.
• Orchestrate Dagster assets, monitor production runs, handle failures and automate alerts so pipelines remain traceable.
• Publish technical and business metadata to DataHub, maintain end-to-end lineage and expose Gold datasets for Power BI reporting.

Your profile

Essential skills

• Apply Python, SQL, Pandas and Apache Spark from a data engineering perspective to build and troubleshoot production pipelines.
• Demonstrate hands-on dbt-core capability across models, snapshots, macros, tests and profiles.
• Understand data architecture, lakehouse and datalake patterns, including Bronze, Silver and Gold design.
• Use Kafka for event-driven ingestion and work with relational platforms such as MSSQL and PostgreSQL, including advanced SQL and CDC.
• Work with Delta Lake features including ACID transactions, time travel, OPTIMIZE and VACUUM.
• Use Dagster or equivalent orchestration concepts and work comfortably with Linux, OpenShift and PowerShell development environments.
• Apply data governance and metadata management practices with DataHub or equivalent tools, including lineage and technical documentation.
• Communicate technical decisions clearly, work autonomously on complex multi-domain pipelines and collaborate effectively with architecture and BI colleagues.
• Hold Belgian nationality, which is required for this programme.