Data Engineer I - PySpark, ETL & Dashboards (Hybrid)

Manchester Digital; Manchester; Hybrid; GBP 35,000 - 43,000; Full time

Benefits offered by this job

Hybrid Working

Civil service pension (28.97%)

Generous leave package (24–31 days)

Job summary

DWP. Digital with Purpose is seeking a Data Engineer to shape the future of data in Debt Solutions. You will design, build and maintain data products supporting analytics and operational decisions, while recovering around £3 billion of debt annually and protecting vulnerable citizens.

Working in an agile team, you will develop ETL pipelines in cloud environments, transform data from multiple sources, and deliver user-facing dashboards and reports that provide trusted insights to stakeholders.

Qualifications

• Experience developing and managing PySpark workflows within AWS environments.
• Knowledge of AWS Lambda and event-driven solutions to support automated data processing.
• Experience provisioning and supporting cloud infrastructure using Amazon EC2 and automation tools.
• Understanding of Agile ways of working and experience contributing to CI/CD delivery practices.
• Ability to optimise data processing, storage, and queries to improve performance and data quality.
• Experience designing and automating ETL pipelines using PySpark and Spark SQL. A strong full stack development background with technologies that include .NET 4.8, C#, JavaScript, C++, and Java.

Responsibilities

• Design, build and maintain data products to support analytics, business intelligence and operational decision-making.
• Work in a multidisciplinary agile team to transform critical business data into trusted insights.
• Develop and operate ETL pipelines in cloud-based environments, integrating data from diverse source systems.
• Create reusable, production-ready solutions using SQL and modern data engineering practices to enable insights and dashboards.

Skills

PySpark

AWS Lambda

EC2

CI/CD

SQL processing

ETL pipelines

Spark SQL

Full stack development

Tools

.NET 4.8

C#

JavaScript

C++

Java

Job description

DWP. Digital with Purpose is seeking a Data Engineer to shape the future of data in Debt Solutions. You will design, build and maintain data products supporting analytics and operational decisions, while recovering around £3 billion of debt annually and protecting vulnerable citizens.

Working in an agile team, you will develop ETL pipelines in cloud environments, transform data from multiple sources, and deliver user-facing dashboards and reports that provide trusted insights to stakeholders.