Senior Data Engineer & Team Lead, Azure Pipelines

DWP Digital; United States; Hybrid; USD 91.000 - 101.000; Vollzeit

Benefits dieser Stelle

Hybrid working

Pension 28.97%

26–31 days holiday

Digital allowance up to £7,459

Flexi-time

Professional development

Zusammenfassung

DWP Digital is seeking a Senior Data Engineer to design, build and enhance cloud-based data pipelines and platforms that empower decision-making across the organisation. You will lead technically, work with stakeholders, and drive best practices across data governance and agile delivery.

You will design scalable ETL/ELT solutions using Python and PySpark, collaborate with multiple business areas across the UK, and mentor other engineers to raise capability within the team.

Qualifikationen

• Design, build, and enhance scalable ETL/ELT pipelines using Python and PySpark.
• Transform data into trusted datasets for insights and decision-making.
• Lead technical teams and shape technical direction for data platforms.
• Work with stakeholders to understand complex requirements and deliver cloud-based data solutions.
• Champion Agile delivery, data governance, and operational excellence.

Aufgaben

• Design and deliver scalable ETL/ELT pipelines with PySpark and Python.
• Collaborate with stakeholders to translate requirements into data products.
• Lead teams through large-scale data and platform transformations.
• Ensure data security, accessibility, and maintainability across platforms.
• Drive automation and performance improvements in data pipelines.

Kenntnisse

Python

ETL/ELT design

Data modeling

Stakeholder engagement

Team leadership

Cloud data engineering

Tools

Azure

PySpark

Spark SQL

Jobbeschreibung

DWP Digital is seeking a Senior Data Engineer to design, build and enhance cloud-based data pipelines and platforms that empower decision-making across the organisation. You will lead technically, work with stakeholders, and drive best practices across data governance and agile delivery.

You will design scalable ETL/ELT solutions using Python and PySpark, collaborate with multiple business areas across the UK, and mentor other engineers to raise capability within the team.