Data Engineer

Venn Group; Lancashire; On-site; GBP 40,000 - 60,000; Full time

Job summary

A public sector organisation in Lancashire is seeking a Data Engineer to enhance their BI function. In this permanent role, you will create and maintain data solutions, develop performance-optimised ETL pipelines, and contribute to the Data Warehouse and BI reporting environment. Ideal candidates will have experience with T-SQL, Python, and Azure Data Factory, alongside a keen eye for data quality assessments.

Qualifications

• Experience in creating and maintaining data solutions for KPI reporting.
• Knowledge of developing scalable, performance-optimised ELT/ETL pipelines.
• Familiarity with maintaining stable, consolidated database platforms for GIS.

Responsibilities

• Support BI reporting team in data solutions.
• Develop and maintain ELT/ETL pipelines.
• Contribute to Data Warehouse and BI reporting environment.
• Implement and manage data flows.
• Automate data quality assessments.

Skills

T-SQL

Python

C#

ADO

PySpark

Jupyter Notebooks

Tools

Azure Data Factory

SSIS

Job description

We’re working with a well-known public sector organisation in Lancashire who are looking to appoint a Data Engineer on a permanent basis. This role sits within a growing Information & BI function and offers the chance to make a real impact across their data landscape.

The role will involve:

• Supporting the BI reporting team by creating and maintaining data solutions for KPI reporting.
• Developing scalable, performance-optimised ELT/ETL pipelines using T-SQL, Python, ADO, C#, PySpark and Jupyter Notebooks.
• Working with the Gazetteer and GIS teams to maintain stable, consolidated database platforms for mapping and GIS systems.
• Contributing to the development and maintenance of a Data Warehouse and organisation-wide BI reporting environment.
• Implementing and managing data flows with tools such as Azure Data Factory and SSIS.
• Working closely with the Business Systems Manager to automate data quality assessments and produce data quality metrics.
• Helping shape best practice through improved development standards, documentation and continuous improvement.
• Thoroughly testing data engineering solutions (unit, integration and UAT).
• Any experience with Microsoft Fabric would be extremely advantageous