Senior Data Engineer — Cloud ETL/PySpark (Azure)

Women in Data®; Greater London; Hybrid; GBP 68.000 - 83.000; Vollzeit

Benefits dieser Stelle

Hybrid working

Flexible hours

Civil service pension with 28.97%

Zusammenfassung

DWP Digital is seeking a Senior Data Engineer to design and deliver cloud-based data solutions on Azure, leveraging PySpark and Spark SQL. You will lead technical teams, shape data pipelines and ensure trusted data fuels informed decision-making across the organisation.

You will work across multiple locations with a hybrid working model, gain exposure to cutting-edge data platforms, and benefit from a competitive package and ongoing professional development.

Qualifikationen

• Proven experience designing and delivering complex ETL/ELT pipelines.
• Ability to communicate complex technical concepts to both technical and non-technical audiences.
• Strong understanding of cloud and distributed data processing.
• Experience leading the resolution of data, platform and pipeline issues.

Aufgaben

• Design, build and enhance scalable ETL/ELT pipelines using Python and PySpark.
• Lead teams through large-scale data and platform transformations and develop capability.
• Collaborate with stakeholders to transform complex requirements into trusted datasets.
• Drive automation, improve platform performance and ensure data governance.

Kenntnisse

ETL/ELT pipelines

Cloud data processing

Team leadership

Stakeholder communication

Tools

PySpark

Spark SQL

Python

Azure

Jobbeschreibung

DWP Digital is seeking a Senior Data Engineer to design and deliver cloud-based data solutions on Azure, leveraging PySpark and Spark SQL. You will lead technical teams, shape data pipelines and ensure trusted data fuels informed decision-making across the organisation.

You will work across multiple locations with a hybrid working model, gain exposure to cutting-edge data platforms, and benefit from a competitive package and ongoing professional development.