Data Engineer - Waverley Court

City of Edinburgh Council; City of Edinburgh; On-site; GBP 48,000 - 56,000; Full time

Job summary

City of Edinburgh Council is seeking a Data Engineer to partner with the Lead Data Engineer to design and build a high-impact data platform using Microsoft Fabric in Edinburgh. Your first project will be leading the integration of multi-source sensor data into our data analytics platform.

You will design, build and maintain our data infrastructure, ensuring the efficient processing, storage and analysis of large datasets for specific projects while adhering to governance and security standards.

Responsibilities

• Lead on designing scalable data pipelines to ETL data into the Microsoft Fabric Platform.
• Collaborate with cross-functional teams to translate data requirements into technical solutions.
• Develop and maintain data models, schemas, and structures for analytics and reporting.
• Ensure data governance, privacy, and security compliance across Data Analytics Platform.
• Document data engineering processes and workflows to enable reproducibility.

Skills

Data pipelines

ETL

Data modeling

Tools

Microsoft Fabric

Job description

Data Engineer
Waverley Court

Salary: £47,639 - £56,392
Hours: 36 per week, 52 weeks

We are looking for a Data Engineer to partner with our Lead Data Engineer to design and build a high-impact solution. Your first project will be leading the integration of multi-source sensor data into our data analytics platform, Microsoft Fabric.

The Data Engineer will be responsible for designing, building, and maintaining our data infrastructure, ensuring the efficient and reliable definition, processing, storage, and analysis of large datasets for specific projects.

The Postholder work with the Lead Data Engineer to develop and, manage our Data Analytics Platform (Microsoft Fabric) and will lead cross-functional project teams across the Council to deliver robust data projects that support the delivery of a roadmap of projects and data products in line with the Council's strategic priorities and outcomes.

They will have the expertise to ensure the delivery of high-quality data products that meet the needs of our stakeholders and enable effective decision-making.

Responsibilities:

• Lead on projects to the design and implement scalable, efficient, and robust data pipelines to extract, transform, and load (ETL) data from various sources into and the Microsoft Fabric Platform
• Lead, engage and collaborate with cross-functional teams, to understand their data requirements and translate them into technical solutions
• Lead the development and maintenance of data models, schemas, and structures that optimize data storage and retrieval for analytics and reporting purposes.
• Working with the Lead Data Engineer , our Senior Information analysts and key service stakeholders to identify ways to improve data reliability, efficiency and quality
• Work across the Council to extract, clean, combine and store data in a way that allows it to be exploited and as an asset to help service delivery and decision making.
• Uphold data governance policies and standards, ensuring compliance with data privacy and security regulations
• Lead on documenting data engineering processes, workflows, and technical specifications to facilitate knowledge sharing and ensure reproducibility of analyses.

We are committed to a diverse and inclusive workforce where everyone feels valued and able to be their best. We particularly encourage applications from women for senior roles, as well as people from minority ethnic backgrounds, people with disabilities or neurodivergent people, care experienced people, carers and LGBT+ people across all levels of the organisation, all of whom are currently underrepresented. All applicants will be considered fairly based on skills and experience. Disabled and care-experienced applicants who meet the minimum job criteria will be guaranteed an interview.

Find out more about our commitment to equalities https://www.edinburgh.gov.uk/work-us/inclusive-edinburgh/4

As part of our goal to improve organisational culture and create a great place to work together for the people of Edinburgh, we want to make sure we're recruiting the best people. We're interested not only in your skills and experience but also in your approach to work. Therefore, part of our interview process will be an assessment of how you would bring Our Behaviours of Respect, Integrity and Flexibility into your ways of working.

You can find out more on Our Behaviours web page Our Behaviours - The City of Edinburgh Council

Our salary range typically reflects the initial starting salary and annually increases until it reaches the top of the range.

Happy to talk flexible working.

Follow us on X at @edincounciljobs

View Data Engineer Job Description

View Data Engineer Person Specification