Lead Data Engineer - Hybrid, Build Trusted Data Services

National Highways; England; Hybrid; GBP 70,000 - 110,000; Full time

Benefits offered by this job

Hybrid role

Job summary

National Highways is strengthening its Data Team within the Digital Services function to deliver data products and enterprise data capabilities across the transport network. The Lead Data Engineer will design, build and govern data pipelines, ensuring reliable, performant services that support operational and strategic decisions.

The role collaborates with stakeholders, embedding data governance and enabling consumption via Databricks, Power BI and ThoughtSpot.

Qualifications

• Experience building and deploying data pipelines and data integration solutions within a collaborative delivery environment, applying engineering standards, automation, testing and governance practices throughout the development lifecycle.
• Experience analysing source systems and complex datasets, bringing together data from multiple sources and applying data modelling or related design techniques to create reusable data assets that support analytics, reporting and business outcomes.
• Experience with scripting CLI, Python and SQL on enterprise cloud data platforms (Azure Databricks, Azure Data Factory) preferred.
• Experience communicating effectively with technical and non-technical stakeholders, managing expectations and ownership of delivery outcomes.
• Experience operating and improving production data pipelines and data products, including monitoring, root cause analysis, performance and cost optimisation.

Responsibilities

• Deliver the Data as a Service (DaaS) roadmap by onboarding enterprise data, improving trust in data and expanding the organisation's analytical and reporting capabilities.
• Design, build and enhance data pipelines, integrations and reusable data assets that bring together transactional, asset, traffic, event and business data within the DaaS platform.
• Collaborate with stakeholders across the organisation to understand analytical needs, develop user stories and deliver trusted data products that support decision-making. Enable consumers through Databricks, Power BI and ThoughtSpot.
• Embed data governance, quality and assurance across the data lifecycle, ensuring compliance with policy, responsible use of AI and metadata management via Unity Catalog.
• Support operation and continuous improvement of production data pipelines, data products and platform capabilities, improving reliability, performance, cost efficiency and user experience.
• Promote data engineering best practices and reusable approaches across the organisation.

Skills

Data pipelines

Data integration

Python

SQL

Stakeholder communication

Governance

Tools

Databricks

Power BI

ThoughtSpot

Unity Catalog

Job description

National Highways is strengthening its Data Team within the Digital Services function to deliver data products and enterprise data capabilities across the transport network. The Lead Data Engineer will design, build and govern data pipelines, ensuring reliable, performant services that support operational and strategic decisions.

The role collaborates with stakeholders, embedding data governance and enabling consumption via Databricks, Power BI and ThoughtSpot.