Hybrid Data Engineering Lead — Cloud Data Platforms

Vanguard; Greater London; Hybrid; GBP 110,000 - 150,000; Full time

Job summary

Vanguard in London seeks a Data Engineering Technical Lead to shape data products and drive analytics across business functions. You will guide a data engineering team, collaborate with product managers, analysts, architects, and stakeholders to deliver scalable data solutions in a hybrid environment.

This role blends hands-on development with strategic leadership, focusing on secure, scalable data platforms and AI-enabled tooling to achieve measurable business outcomes.

Qualifications

• Deep expertise designing and building scalable data pipelines and data products in cloud environments.
• Hands-on with Databricks, Spark, SQL, and Python for data processing.
• Experience developing and optimizing batch and real-time data processing solutions.
• Knowledge of modern data architecture patterns (Kimball, Data Vault, dimensional modelling).
• Experience Infrastructure as Code using Terraform or CloudFormation.
• Experience leading technical delivery teams and mentoring data engineers.
• Strong stakeholder management translating business requirements into scalable technical solutions.
• Experience implementing data governance, data quality, security, and operational controls.

Responsibilities

• Write ETL processes, design databases, and develop tools for real-time and offline analytics.
• Troubleshoot data integrity and integrate large-scale data from multiple sources for insights.
• Translate business specs into design, write complex programs, and document code.
• Partner with internal clients to understand business needs and design data solutions.
• Lead all phases of solution development and communicate with both clients and junior staff.
• Educate and mentor junior data engineers and contribute to engineering standards.
• Test and implement new releases and coordinate with vendors for production readiness.
• Apply AI-assisted development tools to improve engineering productivity and quality.

Skills

Data pipelines

Databricks

Spark

SQL

Python

Cloud environments

Data products

Lead teams

Stakeholder management

Data governance

CI/CD

Agile

Product delivery

AI-assisted development

AWS

Data mesh

MLOps

Generative AI

Distributed teams

Terraform

CloudFormation

Tools

Terraform

CloudFormation

Job description

Vanguard in London seeks a Data Engineering Technical Lead to shape data products and drive analytics across business functions. You will guide a data engineering team, collaborate with product managers, analysts, architects, and stakeholders to deliver scalable data solutions in a hybrid environment.

This role blends hands-on development with strategic leadership, focusing on secure, scalable data platforms and AI-enabled tooling to achieve measurable business outcomes.