Senior Data Engineer

Perch Group; Lancashire; Hybrid; GBP 65,000 - 80,000; Full time

Benefits offered by this job

Hybrid work

Blackpool office

20% bonus

Formal qualifications sponsorship

Wellbeing benefits

Job summary

Perch Group in the UK is seeking a Senior Data Engineer to industrialise our data platform with metadata‑driven frameworks and repeatable patterns. You will design scalable ingestion and publishing pipelines using ADF, Databricks, and T-SQL, while raising engineering standards across the team.

You'll mentor engineers, advance CI/CD and data governance, and partner with product teams to deliver enterprise‑scale data solutions in a hybrid, Blackpool‑based role with opportunities to gain formal

Qualifications

• 5+ years advanced T-SQL with strong optimisation and engineering‑oriented design patterns.
• 5+ years ADF: parameterisation, custom frameworks, and template‑based orchestration.
• 5+ years Databricks including PySpark optimisation, Delta Lake architecture.
• Experience building metadata‑driven or config‑driven frameworks, not just talking about them.
• 5+ years data modelling, including dimensional models for enterprise‑scale consumption.
• End‑to‑end delivery experience across data warehouse or data platform programmes.
• Experience with dynamic schema detection, inferred metadata processing, or multi‑tenant ingestion patterns.
• Advanced orchestration approaches (Databricks Workflows, Functions, event‑driven patterns).
• Practical CI/CD experience using Azure DevOps for data engineering.
• Experience in regulated industries with quality, lineage, and governance requirements.
• Understanding of data management frameworks (DAMA, DCAM, etc.).

Responsibilities

• Design and implement metadata‑driven and config‑driven frameworks to replace one‑off pipelines with reusable, scalable patterns.
• Develop templates, orchestration patterns, and reusable components that minimise manual engineering effort.
• Introduce engineering standards prioritising automation, maintainability, and lifecycle governance.
• Build and optimise robust ingestion, transformation, and publishing pipelines using ADF, Databricks (PySpark/Delta Lake) and T-SQL—embed automation from the start.
• Assist in maturing the enterprise data warehouse architecture and data models.
• Ensure designs align with best practices around performance, logging, monitoring, and quality.
• Partner with architects and product teams to identify high‑value automation opportunities.
• Mentor engineers and uplift engineering discipline through coaching, patterns, and code reviews.
• Strengthen DevOps, CI/CD, and automated testing approaches for data workloads.

Skills

T-SQL

ADF

Databricks

Metadata-driven

Data modelling

CI/CD

Azure DevOps

Data governance

Tools

PySpark

Delta Lake

SQL Server

Job description

Perch Group are searching for a Senior Data Engineer.

At Perch Group, our vision is clear: to lead the UK debt purchase and collection industry by harnessing cutting‑edge technology to drive ethical, efficient, and data‑driven debt resolution.

Our annual mission is to empower hundreds of thousands of customers to positively engage with and resolve their outstanding debts. We do this through an empathetic and customer‑centric approach that is at the heart of our success.

Could this be the next step in your career?

We’re hiring a Senior Data Engineer to drive the next evolution of our data platform by introducing metadata‑driven, configurable, and repeatable engineering frameworks. This role is for someone who has outgrown building pipelines one at a time and is ready to industrialise how a modern data platform operates.

Your focus will be on designing patterns, frameworks, and automation that fundamentally change our speed and quality of delivery. This is an engineering leadership position for someone who can solve problems at the framework and architecture level.

So, what will your day‑to‑day look like?

• Design and implement metadata‑driven and config‑driven frameworks that replace one‑off pipelines with reusable, scalable patterns.
• Develop templates, orchestration patterns, and reusable components that minimise manual engineering effort.
• Introduce engineering standards that prioritise automation, maintainability, and lifecycle governance.

Core Engineering Delivery

• Build and optimise robust ingestion, transformation, and publishing pipelines using ADF, Databricks (PySpark/Delta Lake) and T‑SQL — with automation principles embedded from the start.
• Assist in maturing the enterprise data warehouse architecture and data models.
• Ensure all designs align with best practices around performance, optimisation, logging, monitoring, and quality.
• Partner with architects and product teams to identify high‑value automation opportunities.
• Mentor engineers and uplift engineering discipline through coaching, patterns, and code reviews.
• Strengthen DevOps, CI/CD, and automated testing approaches for data workloads.

Does this sound like you?

We are looking for depth, not surface‑level exposure. Please apply only if you have hands‑on experience in the following:

• 5+ years advanced T‑SQL with strong optimisation, dynamic SQL, and engineering‑oriented design patterns.
• 5+ years ADF: parameterisation, custom frameworks, and template‑based orchestration (not just building pipelines).
• 5+ years Databricks including PySpark optimisation, Delta Lake architecture, and DABs
• Demonstrable experience building metadata‑driven or config‑driven frameworks — not just talking about them.
• 5+ years data modelling, including designing dimensional models that support enterprise‑scale consumption.
• End‑to‑end delivery experience across data warehouse or data platform programmes.
• Experience with dynamic schema detection, inferred metadata processing, or multi‑tenant ingestion patterns.
• Advanced orchestration approaches (Databricks Workflows, Functions, event‑driven patterns).
• Practical CI/CD experience using Azure DevOps for data engineering (not just familiarity).
• Experience in regulated industries where quality, lineage, and governance matter.
• Understanding of data management frameworks (DAMA, DCAM, etc.).

The Application Timeline

• A first stage video call with the internal recruitment team (15 minutes)
• A SQL and ADF technical test either in person or over Zoom (up to 2 hours)
• A face‑to‑face interview with the hiring manager (1 hour)

Typically, the average successful applicant will be within this timeline for 2-3 weeks. Please note we will close this role once we have enough applications for the next stages therefore you should submit your application asap to avoid any disappointment.

If you do not receive a response after 3 weeks of applying, please assume you have been unsuccessful as we may experience a high volume of applications.

Why you should be EXCITED to apply…

• £65,000 + up to 20% of your annual salary, paid as a bonus.
• This role is hybrid and based at our Blackpool office.
• 37.5 hours per week. We offer flexible and hybrid working between our core hours of 8am- 6pm, Monday to Friday.
• The opportunity to complete formal qualifications and learn on the job in a successful, growing organisation.
• And many more benefits to support your wellbeing and professional development.

We’re an equal opportunity employer. All applicants will be considered for employment without attention to age, ethnicity, religion, sex, sexual orientation, gender identity, family or parental status, national origin, or veteran, neurodiversity or disability status.

If you have any questions or suggestions of how we can assist you in your application due to disability or personal reasons, please email [e-mail].

PLEASE NOTE - All new employees must undergo a full DBS and Credit Check upon acceptance of a job offer with Perch Group.