Data Engineering Manager

SF Technology Partners; Greater Manchester; On-site; GBP 90,000 - 120,000; Full time

Job summary

SF Technology Partners is seeking a Data Engineering Manager to lead an established data engineering team and steer the evolution of a modern cloud data platform.

The role sits at the intersection of engineering leadership and platform architecture, requiring hands-on technical knowledge and the ability to influence strategy and delivery.

Qualifications

• Progressed from data engineering into technical leadership without losing engineering expertise.
• Strong experience with Python or PySpark and modern data platforms.
• Proven ability to shape architecture and influence stakeholders.

Responsibilities

• Lead and develop a team of Data Engineers.
• Own the technical direction and architecture of the data platform.
• Set engineering standards and best practices.
• Review designs and challenge technical decisions.
• Improve CI/CD, testing, monitoring and data quality.
• Build scalable data ingestion and transformation processes.
• Collaborate with Head of Data to shape the platform roadmap.
• Communicate recommendations to senior and non-technical stakeholders.
• Develop a platform to support analytics, AI and data products.

Skills

Python / PySpark

Databricks

Lakehouse architectures

CI/CD practices

Lead Data Engineers

Stakeholder communication

Tools

Databricks

Azure

CI/CD pipelines

Job description

I’m recruiting for a Data Engineering Manager to join a large organisation that has made a significant investment in modernising its data capability.

This is a role for someone who sits comfortably between engineering leadership and data platform architecture.

You'll inherit an established Data Engineering team and take responsibility for how they engineer, develop and evolve a modern cloud data platform. You'll manage and develop the team, but this isn't a role for someone who has moved completely away from technology.

They need someone who can challenge an architecture, understand the engineering underneath it and make clear recommendations about how the platform should evolve.

What you'll be responsible for

• Leading and developing a team of Data Engineers
• Owning the technical direction and architecture of the data platform
• Setting engineering standards and ensuring good development practices
• Reviewing designs and challenging technical decisions
• Improving CI/CD, testing, monitoring, data quality and deployment processes
• Building scalable and reliable approaches to data ingestion and transformation
• Working closely with the Head of Data to shape the longer-term platform roadmap
• Communicating technical recommendations clearly to senior and non-technical stakeholders
• Developing a platform capable of supporting future analytics, AI and data products

What we're looking for

You’ll ideally have progressed through a Data Engineering background and moved into technical leadership or management without losing your engineering knowledge.

We're particularly interested in experience across:

• Python / PySpark
• Microsoft Fabric and/or Databricks
• Lakehouse / Medallion architectures
• CI/CD and modern software engineering practices
• Leading and developing Data Engineers
• Working with senior business and technology stakeholders

Microsoft Fabric experience is advantageous rather than essential. Strong Azure/Databricks candidates will absolutely be considered.

The important thing is that you understand good Data Engineering principles and can apply them at platform level – not simply manage delivery.

This could suit an existing Data Engineering Manager, Data Engineering Lead, Data Platform Lead, Principal Data Engineer or hands-on Head of Data Engineering looking for genuine ownership of a modern data environment.