Senior Data Engineer

Perch Group; Lancashire; Hybrid; GBP 65,000 - 78,000; Full time

Benefits offered by this job

Bonus up to 20%

Hybrid work

Blackpool office

Job summary

Perch Group is seeking a Senior Data Engineer to industrialise our data platform with metadata-driven, configurable frameworks. You will design reusable patterns, templates and orchestration to replace bespoke pipelines, while embedding automation and governance across pipelines.

You will work on ingestion, transformation, and publishing using Azure Data Factory, Databricks, and Delta Lake, shaping enterprise data models and DW architecture in a regulated, growth-focused environment.

Qualifications

• 5+ years T-SQL with optimisation and engineering patterns.
• 5+ years ADF parameterisation and orchestration patterns.
• 5+ years Databricks including PySpark and Delta Lake.
• Experience building metadata-driven or config-driven frameworks.
• 5+ years data modelling for enterprise-scale consumption.
• End-to-end delivery across data warehouse projects.
• Experience with dynamic schema detection or multi-tenant ingestion.
• Advanced orchestration (Databricks Workflows, functions, event patterns).
• CI/CD experience using Azure DevOps for data engineering.
• Experience in regulated industries with governance focus.
• Knowledge of DAMA/DCAM data management frameworks.

Responsibilities

• Design and implement metadata-driven frameworks replacing one-off pipelines.
• Develop templates and reusable components to minimise manual work.
• Introduce automation, maintainability, and lifecycle governance.

Skills

T-SQL optimization

Databricks (PySpark)

Metadata-driven design

Data modelling

CI/CD (Azure DevOps)

Tools

Azure Data Factory (ADF)

Databricks

Delta Lake

PySpark

Job description

Perch Group are searching for a Senior Data Engineer.

At Perch Group, our vision is clear: to lead the UK debt purchase and collection industry by harnessing cutting‑edge technology to drive ethical, efficient, and data‑driven debt resolution.

Our annual mission is to empower hundreds of thousands of customers to positively engage with and resolve their outstanding debts. We do this through an empathetic and customer‑centric approach that is at the heart of our success.

Could this be the next step in your career?

We’re hiring a Senior Data Engineer to drive the next evolution of our data platform by introducing metadata‑driven, configurable, and repeatable engineering frameworks. This role is for someone who has outgrown building pipelines one at a time and is ready to industrialise how a modern data platform operates.

So, what will your day‑to‑day look like?

• Design and implement metadata‑driven and config‑driven frameworks that replace one‑off pipelines with reusable, scalable patterns.
• Develop templates, orchestration patterns, and reusable components that minimise manual engineering effort.
• Introduce engineering standards that prioritise automation, maintainability, and lifecycle governance.

Core Engineering Delivery

• Build and optimise robust ingestion, transformation, and publishing pipelines using ADF, Databricks (PySpark/Delta Lake) and T‑SQL — with automation principles embedded from the start.
• Assist in maturing the enterprise data warehouse architecture and data models.
• Ensure all designs align with best practices around performance, optimisation, logging, monitoring, and quality.
• Partner with architects and product teams to identify high‑value automation opportunities.
• Mentor engineers and uplift engineering discipline through coaching, patterns, and code reviews.
• Strengthen DevOps, CI/CD, and automated testing approaches for data workloads.

Does this sound like you?

We are looking for depth, not surface‑level exposure. Please apply only if you have hands‑on experience in the following:

• 5+ years advanced T‑SQL with strong optimisation, dynamic SQL, and engineering‑oriented design patterns.
• 5+ years ADF: parameterisation, custom frameworks, and template‑based orchestration (not just building pipelines).
• 5+ years Databricks including PySpark optimisation, Delta Lake architecture, and DABs
• Demonstrable experience building metadata‑driven or config‑driven frameworks — not just talking about them.
• 5+ years data modelling, including designing dimensional models that support enterprise‑scale consumption.
• End‑to‑end delivery experience across data warehouse or data platform programmes.
• Experience with dynamic schema detection, inferred metadata processing, or multi‑tenant ingestion patterns.
• Advanced orchestration approaches (Databricks Workflows, Functions, event‑driven patterns).
• Practical CI/CD experience using Azure DevOps for data engineering (not just familiarity).
• Experience in regulated industries where quality, lineage, and governance matter.
• Understanding of data management frameworks (DAMA, DCAM, etc.).

The Application Timeline

• A first stage video call with the internal recruitment team (15 minutes)
• A SQL and ADF technical test either in person or over Zoom (up to 2 hours)
• A face‑to‑face interview with the hiring manager (1 hour)

If you do not receive a response after 3 weeks of applying, please assume you have been unsuccessful as we may experience a high volume of applications.

Why you should be EXCITED to apply…

• £65,000 + up to 20% of your annual salary, paid as a bonus.
• This role is hybrid and based at our Blackpool office.
• 37.5 hours per week. We offer flexible and hybrid working between our core hours of 8am- 6pm, Monday to Friday.
• The opportunity to complete formal qualifications and learn on the job in a successful, growing organisation.
• And many more benefits to support your wellbeing and professional development.

We’re an equal opportunity employer. All applicants will be considered for employment without attention to age, ethnicity, religion, sex, sexual orientation, gender identity, family or parental status, national origin, or veteran, neurodiversity or disability status.

If you have any questions or suggestions of how we can assist you in your application due to disability or personal reasons, please email [e-mail].

PLEASE NOTE - All new employees must undergo a full DBS and Credit Check upon acceptance of a job offer with Perch Group.