Data Engineer - Waverley Court

The City of Edinburgh Council; Easter Howgate; On-site; GBP 48,000 - 56,000; Full time

Job summary

The City of Edinburgh Council is seeking a Data Engineer to partner with the Lead Data Engineer and design a high-impact data platform solution. This role will lead integration of multi-source sensor data into the Microsoft Fabric analytics platform and manage scalable data pipelines for critical projects across the Council.

You will design and maintain data infrastructure, ensure quality and governance, and collaborate with stakeholders to deliver data products that support strategic priorities

Qualifications

• Knowledge of designing, building, and maintaining data pipelines for analytics.
• Experience translating data requirements into technical solutions for cross-functional teams.
• Solid understanding of data models, schemas, and data storage for reporting.

Responsibilities

• Lead on projects to design and implement scalable, robust data pipelines to extract, transform, and load data into the Microsoft Fabric Platform.
• Collaborate with cross-functional teams to understand data requirements and translate them into technical solutions.
• Develop and maintain data models, schemas, and structures to optimise data storage and retrieval for analytics and reporting.
• Identify ways to improve data reliability, efficiency and quality across the Council's data systems.
• Work across the Council to extract, clean, integrate and store data as an asset for service delivery and decision making.
• Uphold data governance policies and standards, ensuring compliance with data privacy and security regulations.
• Document data engineering processes, workflows and specifications to facilitate knowledge sharing.

Skills

Data pipelines

ETL

Microsoft Fabric

Data modelling

Governance

Tools

Microsoft Fabric

Job description

Data Engineer
Waverley Court

Salary: £47,639 - £56,392
Hours: 36 per week, 52 weeks

We are looking for a Data Engineer to partner with our Lead Data Engineer to design and build a high-impact solution. Your first project will be leading the integration of multi-source sensor data into our data analytics platform, Microsoft Fabric.

The Data Engineer will be responsible for designing, building, and maintaining our data infrastructure, ensuring the efficient and reliable definition, processing, storage, and analysis of large datasets for specific projects.

The Postholder work with the Lead Data Engineer to develop and, manage our Data Analytics Platform (Microsoft Fabric) and will lead cross-functional project teams across the Council to deliver robust data projects that support the delivery of a roadmap of projects and data products in line with the Council’s strategic priorities and outcomes.

They will have the expertise to ensure the delivery of high-quality data products that meet the needs of our stakeholders and enable effective decision-making.

Responsibilities:

• Lead on projects to the design and implement scalable, efficient, and robust data pipelines to extract, transform, and load (ETL) data from various sources into and the Microsoft Fabric Platform
• Lead, engage and collaborate with cross-functional teams, to understand their data requirements and translate them into technical solutions
• Lead the development and maintenance of data models, schemas, and structures that optimize data storage and retrieval for analytics and reporting purposes.
• Working with the Lead Data Engineer , our Senior Information analysts and key service stakeholders to identify ways to improve data reliability, efficiency and quality
• Work across the Council to extract, clean, combine and store data in a way that allows it to be exploited and as an asset to help service delivery and decision making.
• Uphold data governance policies and standards, ensuring compliance with data privacy and security regulations
• Lead on documenting data engineering processes, workflows, and technical specifications to facilitate knowledge sharing and ensure reproducibility of analyses.

We are committed to a diverse and inclusive workforce where everyone feels valued and able to be their best. We particularly encourage applications from women for senior roles, as well as people from minority ethnic backgrounds, people with disabilities or neurodivergent people, care experienced people, carers and LGBT+ people across all levels of the organisation, all of whom are currently underrepresented. All applicants will be considered fairly based on skills and experience. Disabled and care-experienced applicants who meet the minimum job criteria will be guaranteed an interview.

Find out more about our commitment to equalities https://www.edinburgh.gov.uk/work-us/inclusive-edinburgh/4

As part of our goal to improve organisational culture and create a great place to work together for the people of Edinburgh, we want to make sure we're recruiting the best people. We’re interested not only in your skills and experience but also in your approach to work. Therefore, part of our interview process will be anassessment of how you would bring Our Behaviours of Respect, Integrity and Flexibility into your ways of working.

You can find out more on Our Behaviours web page Our Behaviours – The City of Edinburgh Council

Our salary range typically reflects the initial starting salary and annually increases until it reaches the top of the range.

Happy to talk flexible working.