Data Engineer

SF Partners; Liverpool City Region; On-site; GBP 55,000 - 65,000; Full time

Benefits offered by this job

Benefits

Job summary

SF Partners is seeking a Data Engineer to help shape its modern data platform in St Helens, with a hybrid model requiring 1–2 days on site. The role focuses on Python, PySpark and Microsoft Fabric to build pipelines and data products on top of a growing Fabric/Azure environment.

You’ll contribute to a team moving away from traditional SQL-based tooling, embracing a code-first approach, notebook development, and distributed processing. A flexible benefits package is offered.

Qualifications

• Strong experience with Python, PySpark and data engineering.
• Experience with modern data platforms and Fabric/Azure is a plus.
• Production-grade data solutions through code and distributed processing.

Responsibilities

• Develop pipelines, transformations and data products on the new Fabric platform.
• Adopt a code-led approach with Python and PySpark in production.
• Collaborate with CI/CD, Git and notebook-based development workflows.
• Ensure data quality, monitoring and governance across the data estate.

Skills

Python

PySpark

Microsoft Fabric

Azure

SQL

Databricks

Notebook development

Git

CI/CD

Tools

Git

CI/CD pipelines

Delta tables

Job description

Data Engineer - Python / PySpark / Microsoft Fabric

£55,000 - £65,000 + Benefits

St Helens - 1-2 days per week

This is an opportunity to join an organisation at a particularly interesting point in the development of its data capability.

The business has invested heavily in modernising a sizeable legacy data estate, moving away from traditional SQL-based platforms and establishing a modern Microsoft Fabric/Azure data environment.

The foundations are now in place.

The next stage is about building on them.

As the Data Engineer, you'll join a growing team responsible for developing the pipelines, transformations and data products that sit on top of the new platform.

Importantly, this isn't a traditional Azure Data Engineer role dominated by ADF, stored procedures and drag-and-drop development.

The team is deliberately moving towards a more code-led engineering approach, with a strong emphasis on Python, PySpark and notebook-based development.

What you’ll be working with

The environment includes:

• Python / PySpark
• Microsoft Fabric
• Spark and notebook development
• Lakehouse architecture
• Medallion / Bronze, Silver & Gold patterns
• Delta tables
• SQL
• Azure
• Git / source control
• CI/CD
• Testing, monitoring and data quality

Experience with Azure Databricks is particularly relevant and transfers well into what the team is building within Fabric.

You don't necessarily need commercial Fabric experience if you've worked in a modern Databricks/Spark/Python engineering environment.

Equally, we're open to strong Python Data Engineers who have built production-grade data solutions through code and are looking to move deeper into PySpark, distributed processing and Fabric.

What makes this different?

The business isn't simply replacing one data warehouse with another.

With the core modernisation underway, attention is moving towards how the organisation uses its data, including more sophisticated data products, automation, AI and Generative AI.

That creates an opportunity to join while there's still a significant amount to build, improve and influence.

We're looking for somebody who enjoys engineering - someone who would rather write code, solve problems and build something properly than spend their career connecting boxes together in a UI.

Salary: £55,000 - £65,000

Hybrid: 1-2 days per week in the St Helens