Senior Data Engineer: Production-Grade Pipelines & Lakehouse

Michael Bailey Associates; Randstad; Hybrid; EUR 73,000 - 80,000; Full time

Benefits offered by this job

Learning budget

Paid training days

Hardware & tech allowance

Pension plan

Team trips

Lease car option

Job summary

Michael Bailey Associates is seeking a Senior Data Engineer to treat pipelines like production software and build scalable data platforms for major enterprises in aviation, banking, and retail. You’ll join a top consultancy, owning critical systems with high availability and modularity.

You will design and build production-grade ETL/ELT pipelines, create Bronze/Silver/Gold lakehouses on Azure or GCP, and automate infrastructure with Terraform, Docker, Kubernetes, and Airflow.

Qualifications

• 6+ years of software/data engineering experience.
• Proficient in Python, SQL and PySpark.
• Fluent English and strong stakeholder management.

Responsibilities

• Build production-grade ETL/ELT pipelines using Python, PySpark, and SQL with CI/CD and tests.
• Design Medallion Lakehouses (Bronze/Silver/Gold) on Azure or GCP.
• Automate infrastructure and workflows using Terraform/Bicep, Docker, Kubernetes, and Apache Airflow.
• Integrate AI tools & LLMs into pipelines to boost engineering efficiency.
• Translate business logic into robust architecture while safeguarding data lineage.

Skills

Python

SQL

PySpark

Cloud & Lakehouse

DevOps

Airflow

Kafka

dbt

Infrastructure as Code

Stakeholder management

English

Tools

Terraform

Docker

Kubernetes

Apache Airflow

dbt

Git

Job description

Michael Bailey Associates is seeking a Senior Data Engineer to treat pipelines like production software and build scalable data platforms for major enterprises in aviation, banking, and retail. You’ll join a top consultancy, owning critical systems with high availability and modularity.

You will design and build production-grade ETL/ELT pipelines, create Bronze/Silver/Gold lakehouses on Azure or GCP, and automate infrastructure with Terraform, Docker, Kubernetes, and Airflow.