Data Engineering Lead - Cloud Pipelines & AI

Vanguard; United Kingdom; Hybrid; GBP 90,000 - 130,000; Full time

Benefits offered by this job

Hybrid work 3 days/week

Job summary

Vanguard Europe is seeking a Data Engineering Technical Lead to shape and deliver data and analytics capabilities across multiple business functions. You will lead a team of data engineers in a modern cloud environment using Databricks, AWS, and IaC, and partner with product managers, analysts, and architects to create scalable data solutions.

You will set engineering best practices, promote AI-enabled development, and drive a high‑performing engineering culture across a growing global team.

Qualifications

• Deep expertise in designing and building scalable data pipelines and data products in cloud environments.
• Hands‑on experience with Databricks, Spark, SQL and Python.
• Experience developing batch and real‑time data processing solutions.
• Knowledge of modern data architectures (Kimball, Data Vault, dimensional modelling) and data product design.
• Infrastructure as Code using Terraform or CloudFormation.
• Experience leading technical delivery teams and mentoring engineers.

Responsibilities

• Write ETL processes and design databases for real‑time and offline analytic processing.
• Ensure data quality, integrity, and documentation; write complex queries and reports.
• Collaborate with product managers, analysts, and stakeholders to implement data solutions.
• Lead all phases of solution development and communicate tech decisions to clients and team members.
• Educate junior engineers and contribute to engineering standards and best practices.
• Test and deploy new releases; resolve issues and elevate software to production.

Skills

Databricks

Spark

SQL

Python

Data pipelines

Data analytics

Tools

AWS

Terraform

CloudFormation

CI/CD

Job description

Vanguard Europe is seeking a Data Engineering Technical Lead to shape and deliver data and analytics capabilities across multiple business functions. You will lead a team of data engineers in a modern cloud environment using Databricks, AWS, and IaC, and partner with product managers, analysts, and architects to create scalable data solutions.

You will set engineering best practices, promote AI-enabled development, and drive a high‑performing engineering culture across a growing global team.