Data Engineer: Build Scalable Pipelines on Fabric (Hybrid)

The City of Edinburgh Council; City of Edinburgh; On-site; GBP 48,000 - 56,000; Full time

Job summary

The City of Edinburgh Council is seeking a Data Engineer to partner with the Lead Data Engineer to design and build a high-impact data analytics solution using Microsoft Fabric. Based at Waverley Court in Edinburgh, you will lead integration of multi-source sensor data and develop scalable data pipelines.

You will design, build and maintain data infrastructure for large datasets, collaborate across the Council to deliver data products, ensure governance and security, and document processes to

Responsibilities

• Lead on projects to design and implement scalable, efficient data pipelines to extract, transform, and load data into the Microsoft Fabric Platform
• Lead, engage and collaborate with cross-functional teams to understand their data requirements and translate them into technical solutions
• Lead the development and maintenance of data models, schemas, and structures that optimize data storage and retrieval for analytics and reporting purposes.
• Work with the Senior Information Analysts and key service stakeholders to identify ways to improve data reliability, efficiency and quality
• Extract, clean, combine and store data to support service delivery and decision making
• Uphold data governance policies and standards, ensuring compliance with data privacy and security regulations
• Document data engineering processes, workflows, and technical specifications to facilitate knowledge sharing

Skills

ETL pipelines

Data modeling

Microsoft Fabric

Data governance

Cross-functional teamwork

Data infrastructure

Tools

Microsoft Fabric

Job description

The City of Edinburgh Council is seeking a Data Engineer to partner with the Lead Data Engineer to design and build a high-impact data analytics solution using Microsoft Fabric. Based at Waverley Court in Edinburgh, you will lead integration of multi-source sensor data and develop scalable data pipelines.

You will design, build and maintain data infrastructure for large datasets, collaborate across the Council to deliver data products, ensure governance and security, and document processes to