Lead Databricks Engineer – Real-Time Data Pipelines

United Utilities; Warrington; Hybrid; GBP 75,000 - 110,000; Full time

Benefits offered by this job

Annual bonus 10.5%

Pension 21%

Wellbeing benefits

Job summary

United Utilities is seeking a Lead Data Engineer in Warrington to own and evolve our Databricks-based data platform for near real‑time river water quality insights.

You will design and optimise scalable batch and streaming pipelines, enforce strong data governance and collaborate with data scientists, analysts and environmental specialists to deliver trusted datasets with reliability and cost efficiency.

Qualifications

• Bachelor’s degree in Computer Science, Data Engineering or a related discipline.
• 5+ years’ experience in data engineering roles, including 3+ years with Databricks.

Responsibilities

• Design, build and optimise Databricks-based data pipelines for scalable batch and streaming processing.
• Ensure near real-time monitoring SLAs and data governance across pipelines.
• Collaborate with data scientists and analysts to deliver well-documented datasets.
• Mentor engineers and promote best practices in CI/CD, testing and version control.
• Monitor, troubleshoot and continuously improve production pipelines for reliability and cost efficiency.

Skills

Databricks

Spark

PySpark

Delta Lake

Unity Catalog

Workflows

ETL/ELT pipelines

Python

SQL

Cloud platforms

Data modelling

Data governance

Airflow

CI/CD

Git workflows

Automated testing

IoT data

Education

Bachelor’s degree in Computer Science

Data Engineering related field

Tools

Databricks

Job description

United Utilities is seeking a Lead Data Engineer in Warrington to own and evolve our Databricks-based data platform for near real‑time river water quality insights.

You will design and optimise scalable batch and streaming pipelines, enforce strong data governance and collaborate with data scientists, analysts and environmental specialists to deliver trusted datasets with reliability and cost efficiency.