Hybrid Data Engineer - AWS PySpark & ETL

Career Choices Dewis Gyrfa Ltd; Newcastle upon Tyne; Hybrid; GBP 44,000 - 52,000; Full time

Benefits offered by this job

Hybrid working

Civil Service Pension

Job summary

DWP Digital is hiring a Data Engineer to join the Management Information team in the UK. You will design, build and optimise scalable data products using PySpark and AWS services, enabling analysts and data scientists to deliver insights that influence strategic decisions.

In this hybrid role, you will contribute to CI/CD practices and collaborate with diverse stakeholders to ensure high data quality, performance and security across large data sets.

Qualifications

• Experience developing and managing PySpark workflows within AWS environments.
• Knowledge of AWS Lambda and event-driven solutions to support automated data processing.
• Experience provisioning and supporting cloud infrastructure using Amazon EC2 and automation tools.
• Understanding of Agile ways of working and experience contributing to CI/CD delivery practices.

Responsibilities

• Design, build, maintain, and optimise scalable data products used by analysts and data scientists.
• Develop and orchestrate PySpark-based ETL pipelines using AWS technologies to process large data volumes.
• Collaborate with stakeholders to translate reporting requirements into robust technical solutions while ensuring data quality and security.

Skills

PySpark

AWS Lambda

EC2

CI/CD

Agile

Data optimisation

Tools

Amazon EMR

Job description

DWP Digital is hiring a Data Engineer to join the Management Information team in the UK. You will design, build and optimise scalable data products using PySpark and AWS services, enabling analysts and data scientists to deliver insights that influence strategic decisions.

In this hybrid role, you will contribute to CI/CD practices and collaborate with diverse stakeholders to ensure high data quality, performance and security across large data sets.