Senior Data Engineer – Azure Pipelines, Hybrid Role

DWP Digital; Birmingham; Ibrido; GBP 68.000 - 76.000; Tempo pieno

Vantaggi offerti da questo lavoro

Hybrid working

Flexible hours

Pension with 28.97% employer contrib.

Training & development

Descrizione del lavoro

DWP Digital, part of the UK Civil Service, is expanding its Data Engineering community. We are seeking an experienced Senior Data Engineer with strong Azure and data-pipeline skills to lead technical teams and deliver modern cloud-based data solutions across multiple sites.

You will design, build and optimize scalable ETL/ELT pipelines, ensure trusted data for decision-making and enforce engineering standards while developing others within the team.

Competenze

• Proven experience designing and delivering complex ETL/ELT pipelines.
• Ability to communicate complex technical concepts to technical and non-technical audiences.
• Strong understanding of cloud and distributed data processing, scalable and maintainable data solutions.
• Experience analyzing requirements, producing clear specifications, and defining testing approaches.
• Leading the resolution of data, platform and pipeline issues across levels.
• Knowledge migrating and transforming data from legacy/on-premise to cloud.
• Expertise automating ETL/ELT pipelines using PySpark and Spark SQL on Azure.

Mansioni

• Design, build and enhance scalable ETL/ELT pipelines using Python and PySpark.
• Collaborate with stakeholders to translate complex requirements into data solutions.
• Manage and optimize data across multiple platforms, ensuring security and accessibility.
• Lead teams through large-scale data and platform transformations and promote best practices.
• Drive automation, improve platform performance and support Agile delivery and data governance.

Conoscenze

ETL/ELT pipelines

PySpark

Spark SQL

Python

Cloud data solutions

Stakeholder communication

Data governance

Strumenti

Azure

Descrizione del lavoro

DWP Digital, part of the UK Civil Service, is expanding its Data Engineering community. We are seeking an experienced Senior Data Engineer with strong Azure and data-pipeline skills to lead technical teams and deliver modern cloud-based data solutions across multiple sites.

You will design, build and optimize scalable ETL/ELT pipelines, ensure trusted data for decision-making and enforce engineering standards while developing others within the team.