Data Engineer

Government Recruitment Service; Newcastle upon Tyne; On-site; GBP 60,000 - 80,000; Full time

Job summary

A government agency in Newcastle upon Tyne is seeking an experienced Data Engineer to join their Data Analytics Service Team. In this role, you will streamline ETL processes, ensure effective data pipelines, and provide leadership within the data team. This position requires strong familiarity with Azure and data engineering practices. The ideal candidate will develop relationships with civil service partners to enhance data services. Attractive role for professionals passionate about data analytics and team leadership.

Qualifications

• Experience in data engineering and ETL processes.
• Leadership skills to manage team dynamics.
• Strong understanding of data analysis methods.

Responsibilities

• Streamline ETL processes for datasets.
• Ensure pipelines execute correctly and propose improvements.
• Develop relationships with key partners across the civil service.

Skills

Data Engineering

ETL processes

Data analysis

Leadership

Azure

Tools

Azure

Databricks

Job description

This role sits within the Data Analytics Service Team. You will be working as our Data Engineer and will be required to review/ design, document, and streamline the ETL processes for the datasets we bring together for analysis and AI projects. We operate in Azure using ABFSS for storage and Databricks Unity catalogue for our data processing. As part of the Senior Management Team, you will ensure new datasets are being added with the correct processes, existing pipelines execute correctly, and proposing changes to these processes going forward in line with best practices.

As part of the senior management of the data analytics team, you will also have leadership responsibilities within the PSFA data team to help us improve our services.

You will work with the platform lead to identify where and how we should continue to build our Azure tenant to best support the ambitions of the PSFA data analytics service going forward. You will develop relationships with key partners across the civil service, including our key data providers and our cloud provider, to identify how we can work together to improve our tenant.