Senior Data Engineer

Templeton & Partners - Innovative & Inclusive Hiring Solutions; Randstad; On-site; EUR 90,000 - 130,000; Full time

Job summary

Templeton & Partners - Innovative & Inclusive Hiring Solutions is seeking a Senior Data Engineer to drive the modernization of our Enterprise Data Platform. This hands-on role focuses on building reusable migration frameworks and accelerators to enable scalable migrations.

You will work at the intersection of data engineering, lakehouse architectures, migration, and AI-enabled automation, delivering production-grade data pipelines and reusable frameworks for cross-team adoption.

Qualifications

• 5+ years of experience in data engineering.
• Experience with Databricks, Apache Spark, Python and SQL.
• Strong understanding of Delta Lake, Lakehouse and Medallion architectures.
• Experience designing and migrating legacy data platforms to cloud Lakehouse environments.
• Experience building reusable data engineering frameworks and accelerators.
• Knowledge of data quality, testing, CI/CD and production readiness.
• Excellent communication with architects, engineers and business stakeholders.

Responsibilities

• Design and build reusable migration frameworks, templates, utilities and accelerators for onboarding workloads onto our EDP.
• Modernize legacy data pipelines and data models into Databricks Lakehouse and Medallion architectures.
• Develop production-grade data pipelines using Databricks, Spark, Python/Scala, SQL and DBT.
• Analyze existing data platforms and propose migration, refactoring and modernization strategies.
• Build automated approaches for data validation, reconciliation, testing, monitoring and performance optimization.
• Establish standards and reusable patterns for migration teams.
• Collaborate with business and engineering teams to map current-state data landscapes.
• Identify opportunities to automate migration activities using GenAI, agents and tooling.
• Contribute to architecture reviews and migration planning.

Skills

Data engineering

Python

SQL

Databricks

Apache Spark

DBT

Tools

Databricks

Apache Spark

Python

SQL

DBT

Delta Lake

Job description

Build the future of our Enterprise Data Platform

We’re looking for a Senior Data Engineer to join our Enterprise Data Platform (EDP) modernization programme.

This is a hands-on role for someone who enjoys solving complex data engineering problems, modernizing legacy platforms and, importantly, building reusable solutions that help other teams migrate faster.

You’ll work at the intersection of Data Engineering, Lakehouse architecture, migration and AI-enabled automation. You won’t just migrate individual pipelines, you’ll create the frameworks, accelerators and engineering patterns that enable migration at scale.

What you’ll do

• Design and build reusable migration frameworks, templates, utilities and accelerators for onboarding workloads onto our EDP.
• Modernize legacy data pipelines and data models into Databricks Lakehouse and Medallion architectures.
• Develop production-grade data pipelines using Databricks, Apache Spark, Python/Scala, SQL and DBT.
• Analyse existing data platforms and recommend practical approaches for migration, refactoring and modernization.
• Build automated approaches for data validation, reconciliation, testing, monitoring and performance optimisation.
• Establish engineering standards, reusable patterns and reference implementations that can be adopted across migration teams.
• Work directly with business and engineering teams to understand their current-state data landscape and guide them through migration.
• Identify opportunities to automate repetitive migration activities using AI, GenAI, agents, scripting and reusable engineering tools.
• Contribute to technical workshops, architecture discussions, design reviews and migration planning.

What we’re looking for

You’re a hands-on senior data engineer who combines strong engineering fundamentals with an interest in modernization and automation.

You’ll ideally have:

• Strong commercial experience in data engineering, ideally 5+ years.
• Significant experience with Databricks, Apache Spark, Python and SQL.
• Strong understanding of Delta Lake, Lakehouse and Medallion architectures.
• Experience designing and optimizing scalable batch and/or streaming data pipelines.
• Practical experience with DBT and modern ELT/data modelling practices.
• Experience migrating or modernizing legacy data platforms, pipelines or warehouses into cloud/Lakehouse environments.
• Experience building reusable frameworks, libraries, templates or engineering accelerators.
• Understanding of data quality, reconciliation, testing, CI/CD and production readiness.
• Strong communication skills and the ability to work directly with architects, engineers and business stakeholders.

You don’t need to be an AI specialist.

What matters is that you’re curious about how GenAI and agentic automation can make data engineering and migration more efficient.

Experience with AI-assisted development, LLMs, coding copilots, AI agents or workflow automation is highly desirable.

You’ll be a great fit:

• Migrating legacy data warehouses to Databricks
• Moving on-premise data workloads to cloud Lakehouse architectures
• Building metadata-driven or reusable data engineering frameworks
• Converting legacy SQL/ETL processes into Spark/DBT
• Automating data quality and reconciliation
• Building developer or migration tooling
• Using GenAI to accelerate engineering or migration activities

If you’re a Senior Data Engineer, Lead Data Engineer, Data Platform Engineer, Databricks Engineer or Data Engineering Consultant who wants to work on large-scale modernization rather than business-as-usual pipeline development, we’d like to hear from you.