Senior Data Engineer — Metadata-Driven Data Platform

Perch Group; Lancashire; Híbrido; GBP 65 000 - 78 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Bonus up to 20%

Hybrid work

Blackpool office

Resumo da oferta

Perch Group is seeking a Senior Data Engineer to industrialise our data platform with metadata-driven, configurable frameworks. You will design reusable patterns, templates and orchestration to replace bespoke pipelines, while embedding automation and governance across pipelines.

You will work on ingestion, transformation, and publishing using Azure Data Factory, Databricks, and Delta Lake, shaping enterprise data models and DW architecture in a regulated, growth-focused environment.

Qualificações

• 5+ years T-SQL with optimisation and engineering patterns.
• 5+ years ADF parameterisation and orchestration patterns.
• 5+ years Databricks including PySpark and Delta Lake.
• Experience building metadata-driven or config-driven frameworks.
• 5+ years data modelling for enterprise-scale consumption.
• End-to-end delivery across data warehouse projects.
• Experience with dynamic schema detection or multi-tenant ingestion.
• Advanced orchestration (Databricks Workflows, functions, event patterns).
• CI/CD experience using Azure DevOps for data engineering.
• Experience in regulated industries with governance focus.
• Knowledge of DAMA/DCAM data management frameworks.

Responsabilidades

• Design and implement metadata-driven frameworks replacing one-off pipelines.
• Develop templates and reusable components to minimise manual work.
• Introduce automation, maintainability, and lifecycle governance.

Conhecimentos

T-SQL optimization

Databricks (PySpark)

Metadata-driven design

Data modelling

CI/CD (Azure DevOps)

Ferramentas

Azure Data Factory (ADF)

Databricks

Delta Lake

PySpark

Descrição da oferta de emprego

Perch Group is seeking a Senior Data Engineer to industrialise our data platform with metadata-driven, configurable frameworks. You will design reusable patterns, templates and orchestration to replace bespoke pipelines, while embedding automation and governance across pipelines.

You will work on ingestion, transformation, and publishing using Azure Data Factory, Databricks, and Delta Lake, shaping enterprise data models and DW architecture in a regulated, growth-focused environment.