Senior Data Engineer

SF Partners; St Helens; Hybrid; GBP 70,000 - 80,000; Full time

Job summary

SF Partners is recruiting a hands-on Senior Data Engineer in the UK to advance a modern data capability built on Python/PySpark, Fabric, and Azure. You will contribute to lakehouse/medallion style designs, with a focus on code-led development and AI-ready data solutions.

You'll influence architecture, coding standards, CI/CD, testing, and best practices while remaining deeply technical and hands-on. The role offers 1–2 days per week in the North West and potential flexibility for exceptional

Qualifications

• Hands-on data engineering experience at Senior level.
• Experience with modern data platforms and cloud native architectures.
• Strong coding standards, testing, and CI/CD practices.

Responsibilities

• Lead design and implementation of data pipelines and models.
• Influence architecture, coding standards, and best practices across the team.
• Create reusable engineering patterns and ensure data quality and observability.
• Collaborate with cross-functional teams to deliver a modern data capability.

Skills

Python

PySpark

Fabric

Spark

Delta

SQL

Azure

Git

CI/CD

Data quality

Job description

I'm recruiting a Senior Data Engineer to join an established organisation continuing a significant modernisation of its data capability.

Over the last couple of years, the business has been moving away from a legacy data estate and establishing a modern Microsoft Fabric/Azure platform.

The next stage is about turning those foundations into a genuinely modern, code-led engineering environment.

This is a hands-on Senior Data Engineer role. They aren't looking for someone who wants to move away from development into pure management.

You'll work across:

• Python / PySpark
• Microsoft Fabric
• Spark and notebook engineering
• Lakehouse / Medallion architecture
• Delta
• SQL
• Azure
• Git / CI/CD
• Automated testing
• Data quality and observability
• Reusable engineering patterns

They're particularly interested in people coming from Databricks/PySpark environments, so extensive Fabric experience isn't essential.

At Senior level, you'll also have the opportunity to influence architecture, technical design, coding standards, CI/CD, testing and engineering best practice, while supporting other engineers and taking ownership of more complex technical problems.

There's still a significant amount to build, and the longer-term direction extends beyond traditional data engineering into AI, Generative AI and intelligent automation.

It's a good opportunity for somebody who wants to remain deeply technical and hands-on, but wants more influence over how a modern data engineering capability develops.

£70,000 - £80,000| North West | 1-2 days per week

Potential flexibility above this for an exceptional candidate