Lead Data Engineer

National Highways; Guildford; Hybrid; GBP 70,000 - 100,000; Full time

Job summary

National Highways is hiring a Lead Data Engineer to design, build and operate scalable data pipelines for the DaaS platform. You will collaborate with stakeholders to turn complex data into trusted, well-governed data products and insights across the organisation.

The role emphasizes DevOps practices, data governance, and a strong focus on enabling analytics via Databricks, Power BI, and ThoughtSpot, with hybrid work requiring two days per week in the office.

Qualifications

• Experience communicating with non-technical stakeholders to gather requirements and translate complex data concepts.
• Ability to understand complex datasets, source systems design, and describe them via data modelling.
• Experience designing, building, testing, and supporting moderate to complex data pipelines in a DevOps environment.
• Strong understanding of the development lifecycle and managing stakeholder delivery expectations.
• Awareness of emergent technologies and experience with the described tech stack.

Responsibilities

• Deliver the Data as a Service (DaaS) roadmap, improving trust in data, building data products, and strengthening the enterprise data platform.
• Design, build, and operate scalable data pipelines from operational systems into DaaS, using Azure DevOps, Databricks, and Azure services.
• Translate analytical and reporting needs into robust data products using Databricks workspaces, Power BI, and ThoughtSpot.
• Embed data governance, quality, and assurance across the data lifecycle, ensuring policy compliance, metadata management, and data quality.
• Define, embed, and champion data engineering best practices across the organisation.

Skills

Data pipelines

DevOps

Stakeholder comms

Data modeling

Azure

Tools

Databricks

Azure Data Factory

Power BI

ThoughtSpot

Unity Catalog

Microsoft Purview

Job description

About the role.

As part of the development of our Digital Services function, we are strengthening internal digital capability within the Data Team. These roles are central to building sustainable in-house expertise that supports the design, development, and delivery of digital solutions across the transport network.

A key focus of this campaign is creating new opportunities, and as a result, we are recruiting Lead Data Engineers. This is a pivotal role within the organisation.

In this role, you will design, build, and support data pipelines within our Data as a Service (DaaS) platform. Working closely with stakeholders, you will bring together traffic, asset, event, and business data, ensuring it is secure, well-structured, and reliable throughout its lifecycle.

You will enable teams across the business to easily access, analyse, and use trusted data for operational insight, reporting, and improved decision-making. This is an opportunity to work with large and complex datasets, contributing to the country\'s growth through a safe and efficient road network.

This is a hybrid role, requiring two days per week in the office, based at any National Highways office location.

• Deliver the Data as a Service (DaaS) roadmap, improving trust in data, building data products, and strengthening the enterprise data platform.
• Design, build, and operate scalable data pipelines from operational systems into DaaS, working as part of a DevOps aligned engineering team using Azure DevOps. The platform is centred on Databricks, with Azure Data Factory for orchestration and Azure services for storage.
• Translate analytical and reporting needs into robust data products by collaborating closely with stakeholders at all levels across the organisation. Develop clear user stories that support analytics and decision making use cases, enabling consumers through Databricks workspaces, Power BI, and ThoughtSpot.
• Embed data governance, quality, and assurance across the full data lifecycle, ensuring compliance with organisational policy, responsible use of AI, strong metadata management, and high data quality standards. Governance is enabled through Unity Catalog and Microsoft Purview.
• Define, embed, and champion data engineering best practices and standards, while driving adoption and effective use of enterprise data capabilities across the organisation.

About you.

• Experience communicating with non-technical stakeholders in support of effective requirement gathering and in translating complex data engineering concepts into clear, accessible language.
• Able to develop an understanding of complex datasets, their source systems design and then describe them through techniques such as data modelling.
• Experience designing, building, testing, and supporting moderate to complex data pipelines within a collaborative (DevOps) team.
• Strong understanding of the development lifecycle ensuring that teams adhere to processes and capable of managing stakeholder\'s expectations for delivery.
• Able to apply industry best practices awareness of emergent technologies and experience with the technology described.

About us.

Here at National Highways, we manage and improve England\'s motorways and major A roads, helping our customers have safer, smoother and more reliable journeys. Our priorities are safety, customers and delivery, and at the core of this, are our values of passion, integrity, safety, teamwork and ownership.

The Digital Services Directorate is helping to reshape the way National Highways operates by delivering digital, data, and technology services across all areas, from frontline to back office, in a modern and efficient manner. Our vision is to develop integrated information and technology that empowers our colleagues and provides real-time information to our customers, as well as integrating with intelligent vehicle and transport systems as they evolve, to enhance journey safety and reliability.

External candidates will be offered a starting salary at the lower end of the pay scale, while current employees will be appointed in accordance with our established pay policy.

We are committed to creating a diverse environment and welcome applicants from all backgrounds.