Data Engineer

TyneStack Ltd; Newcastle upon Tyne; Hybrid; GBP 55,000 - 90,000; Full time

Benefits offered by this job

Hybrid working arrangement

Job summary

TyneStack Ltd is seeking an experienced Data Engineer to join the Newcastle Upon Tyne Data Engineering team. The role focuses on transforming transactional data into scalable analytical datasets using Microsoft Fabric, SQL and PySpark, and designing OLAP models and data pipelines to underpin Power BI reporting and analytics.

You will work closely with Developers, Analytics and Data Science teams to modernise the data platform, migrate legacy reporting solutions and ensure reliable data for

Qualifications

• Strong commercial experience with T-SQL and complex SQL development.
• Hands-on experience with Spark and PySpark for data processing and transformation.
• Strong understanding of OLTP and OLAP architectures.
• Experience designing dimensional data models using facts and dimensions.
• Experience with Microsoft Fabric and wider Azure data technologies.
• Strong knowledge of ETL and ELT processes and data integration.
• Experience developing and optimising production data pipelines.
• Strong understanding of data quality, governance and security principles.
• Azure Data Lake, Data Warehouse and Blob Storage experience.
• Experience supporting Data Science and predictive modelling teams.
• Experience migrating legacy data platforms into Microsoft Fabric.

Responsibilities

• Design and develop OLAP data models, including facts and dimensions, from complex OLTP systems.
• Build, maintain and optimise data pipelines within Microsoft Fabric.
• Develop large-scale data transformations using Spark and PySpark.
• Write, optimise and troubleshoot complex T-SQL queries and procedures.
• Build ETL and ELT pipelines integrating internal and external data sources.
• Migrate legacy reporting data and processes into Microsoft Fabric.
• Design data warehouse models to support reporting and analytical requirements.
• Create engineered datasets and summary tables for Power BI and Analytics teams.
• Develop aggregated datasets to support Data Science and predictive modelling.
• Monitor data quality, pipeline performance, reliability and processing efficiency.

Tools

Microsoft Fabric

Azure data technologies

Spark

PySpark

T-SQL

Data pipelines

OLAP/OLTP

Job description

Data Engineer (Microsoft Fabric / SQL / PySpark)

Location: Newcastle Upon Tyne | Hybrid

Salary: Competitive + Benefits

Type: Full-time, Permanent

Overview

A leading technology and financial services organisation is looking to appoint an experienced Data Engineer to join its growing Data Engineering function.

This is a hands-on engineering role focused on transforming complex transactional data into scalable, high-quality analytical datasets. You will work extensively with Microsoft Fabric, SQL and PySpark, designing OLAP models, developing data pipelines and creating the data structures that underpin Power BI reporting, analytics and Data Science.

Working closely with Developers, Analytics and Data Science teams, you will play an important role in modernising the organisation's data platform, migrating legacy reporting solutions and ensuring reliable data is available for business-critical decision making.

Key Responsibilities

• Design and develop OLAP data models, including facts and dimensions, from complex OLTP systems
• Build, maintain and optimise data pipelines within Microsoft Fabric
• Develop large-scale data transformations using Spark and PySpark
• Write, optimise and troubleshoot complex T-SQL queries and procedures
• Build ETL and ELT pipelines integrating internal and external data sources
• Migrate legacy reporting data and processes into Microsoft Fabric
• Design data warehouse models to support reporting and analytical requirements
• Create engineered datasets and summary tables for Power BI and Analytics teams
• Develop aggregated datasets to support Data Science and predictive modelling
• Monitor data quality, pipeline performance, reliability and processing efficiency

Requirements

• Strong commercial experience with T-SQL and complex SQL development
• Hands-on experience with Spark and PySpark for data processing and transformation
• Strong understanding of OLTP and OLAP architectures
• Experience designing dimensional data models using facts and dimensions
• Experience with Microsoft Fabric and wider Azure data technologies
• Strong knowledge of ETL and ELT processes and data integration
• Experience developing and optimising production data pipelines
• Strong understanding of data quality, governance and security principles
• Azure Data Lake, Data Warehouse and Blob Storage experience
• Experience supporting Data Science and predictive modelling teams
• Experience migrating legacy data platforms into Microsoft Fabric

What's on Offer

• Competitive salary plus bonus and benefits
• Hybrid working arrangement
• Opportunity to work extensively with Microsoft Fabric, SQL and PySpark
• Work on large-scale data transformation and modernisation projects
• Exposure to Analytics, Power BI and Data Science initiatives
• Excellent technical development and career progression opportunities