Hybrid Data Engineer: Databricks, Kafka & Azure (Logistics)

Queen Square Recruitment Ltd; Rotterdam, Den Haag; Hybrid; EUR 70,000 - 100,000; Full time

Job summary

Queen Square Recruitment Ltd in Rotterdam area is seeking a Data Platform Engineer to join a small platform team modernising its data platform and delivering real-time visibility across warehousing, transport and fleet operations.

You will build batch and streaming pipelines using Databricks, PySpark and Delta Lake on Azure, ingesting events from Kafka and Azure Event Hubs and modeling data with dbt for Power BI reporting.

Qualifications

• Strong Python, PySpark and SQL.
• Production experience with Databricks and Delta Lake on Azure.
• Streaming experience with Kafka or Event Hubs.
• dbt in a production setting.
• Terraform and Git based CI/CD.
• Fluent English; Dutch a plus, not essential.

Responsibilities

• Build and maintain batch and streaming pipelines in Databricks using PySpark and Delta Lake.
• Ingest shipment and telematics events from Kafka and Azure Event Hubs into a medallion architecture.
• Integrate data from WMS and TMS platforms (Blue Yonder, Manhattan) plus EDI messages from carriers.
• Model curated layers with dbt for reporting in Power BI.
• Orchestrate workloads with Azure Data Factory and Databricks Workflows.
• Manage infrastructure with Terraform and CI/CD in Azure DevOps.
• Apply governance and access control through Unity Catalog.

Skills

Python

PySpark

SQL

Databricks

Delta Lake

Kafka

Event Hubs

dbt

Terraform

CI/CD

Git

Azure DevOps

Power BI

Tools

Power BI

Job description

Queen Square Recruitment Ltd in Rotterdam area is seeking a Data Platform Engineer to join a small platform team modernising its data platform and delivering real-time visibility across warehousing, transport and fleet operations.

You will build batch and streaming pipelines using Databricks, PySpark and Delta Lake on Azure, ingesting events from Kafka and Azure Event Hubs and modeling data with dbt for Power BI reporting.