Senior Data Engineer - Azure PySpark ETL/ELT (Hybrid)

DWP Digital; Tees Valley; Hybrid; GBP 68,000 - 83,000; Full time

Benefits offered by this job

Hybrid working

Flexible hours

Civil service pension 28.97%

Job summary

DWP Digital is seeking an experienced Senior Data Engineer to design and deliver cloud-based data solutions using Azure, PySpark and Spark SQL. You will lead technical teams, ensuring high-quality, trusted data across platforms.

The role offers hybrid working, flexible hours, and a structured progression path within the DWP Digital Data Engineering community across multiple UK locations.

Qualifications

• Experience designing and delivering complex ETL/ELT pipelines with quality outcomes.
• Ability to communicate complex tech concepts to technical and non-technical audiences.
• Strong cloud/data processing knowledge and scalable data solution design.
• Proven track record leading data engineering teams and projects.

Responsibilities

• Lead design and delivery of modern cloud-based data solutions.
• Build and optimise scalable ETL/ELT pipelines using Python and PySpark.
• Mentor team members and promote engineering best practices.
• Collaborate with stakeholders to translate business requirements into data products.

Skills

ETL Pipelines

PySpark

Spark SQL

Azure

Team leadership

Data platforms

Tools

Azure Data Factory

Databricks

Job description

DWP Digital is seeking an experienced Senior Data Engineer to design and deliver cloud-based data solutions using Azure, PySpark and Spark SQL. You will lead technical teams, ensuring high-quality, trusted data across platforms.

The role offers hybrid working, flexible hours, and a structured progression path within the DWP Digital Data Engineering community across multiple UK locations.