Hybrid Data Engineer – PySpark on AWS

Career Choices Dewis Gyrfa Ltd

Manchester

Hybrid

GBP 46,000 - 57,000

Full time

13 days ago

Get more replies from employers

Send a job-specific resume in minutes.

Benefits offered by this job

Hybrid working

Competitive pension

Generous leave 26–31 days

Flexi-time

Professional development

Public sector recognition

Job summary

DWP is seeking a Data Engineer to join the Management Information (MI) team. You will develop scalable data solutions using AWS technologies to power reporting, analytics, and strategic decision-making.

Based in Manchester, you will design and orchestrate PySpark ETL pipelines with EMR, Lambda, and EC2, collaborating with a broad range of stakeholders to ensure data quality and security, while contributing to Agile delivery and CI/CD practices.

Qualifications

Experience developing and managing PySpark workflows within AWS environments.
Knowledge of AWS Lambda and event-driven solutions to support automated data processing.
Experience provisioning and supporting cloud infrastructure using Amazon EC2 and automation tools.
Understanding of Agile ways of working and experience contributing to CI/CD delivery practices.
Ability to optimise data processing, storage, and queries to improve performance and data quality.

Responsibilities

Design, build, maintain, and optimise scalable data products used by analysts and data scientists.
Develop and orchestrate PySpark-based ETL pipelines using EMR, Lambda, and EC2 to process large datasets.
Collaborate with stakeholders to translate reporting requirements into robust technical solutions.
Support Agile delivery practices and CI/CD automation; maintain technical documentation and conduct code reviews.