Senior Data Engineer

SF Partners; Liverpool City Region; On-site; GBP 70,000 - 80,000; Full time

Job summary

SF Partners is modernising its data capability with a hands-on Senior Data Engineer role in the North West. You will work on Python/PySpark, Fabric and lakehouse patterns to deliver robust data pipelines.

The role emphasizes architectural influence while staying deeply technical and hands-on, with opportunities to push toward AI and intelligent automation in a code-led environment.

Qualifications

• This is a hands-on Senior Data Engineer role requiring strong coding and architectural influence.
• Candidates should have deep PySpark/Databricks experience and modern data stack know‑how.
• Experience with Fabric, lakehouse patterns and CI/CD is highly desirable.

Responsibilities

• Hands-on data engineering across Python, PySpark and SQL in a modern Azure Fabric environment.
• Help shape architecture, coding standards and best practices for data pipelines.
• Build reusable patterns, ensure data quality and observability across the lakehouse.
• Collaborate with engineers to drive AI and automation initiatives.

Skills

Python

PySpark

Microsoft Fabric

Spark

Notebook engineering

Lakehouse architecture

Delta

SQL

Azure

Git / CI/CD

Automated testing

Data quality

Observability

Reusable engineering patterns

Tools

Databricks

Azure DevOps

CI/CD tooling

Job description

I'm recruiting a Senior Data Engineer to join an established organisation continuing a significant modernisation of its data capability.

Over the last couple of years, the business has been moving away from a legacy data estate and establishing a modern Microsoft Fabric/Azure platform.

The next stage is about turning those foundations into a genuinely modern, code-led engineering environment.

This is a hands‑on Senior Data Engineer role. They aren't looking for someone who wants to move away from development into pure management.

You'll work across:

• Python / PySpark
• Microsoft Fabric
• Spark and notebook engineering
• Lakehouse / Medallion architecture
• Delta
• SQL
• Azure
• Git / CI/CD
• Automated testing
• Data quality and observability
• Reusable engineering patterns

They're particularly interested in people coming from Databricks/PySpark environments, so extensive Fabric experience isn't essential.

At Senior level, you'll also have the opportunity to influence architecture, technical design, coding standards, CI/CD, testing and engineering best practice, while supporting other engineers and taking ownership of more complex technical problems.

There's still a significant amount to build, and the longer‑term direction extends beyond traditional data engineering into AI, Generative AI and intelligent automation.

It's a good opportunity for somebody who wants to remain deeply technical and hands‑on, but wants more influence over how a modern data engineering capability develops.

£70,000 - £80,000 North West 1-2 days per week

Potential flexibility above this for an exceptional candidate.