Lead Data Engineer: Architect Data Platforms & Pipelines

Quilter plc; City of Westminster; On-site; GBP 90,000 - 130,000; Full time

Benefits offered by this job

Holiday entitlement and pension

Private Medical Insurance

Life Assurance 4x salary

Income protection

Flexible benefits program

Job summary

Quilter plc leads data engineering with a strong focus on scalable, secure data platforms. The Lead Data Engineer within the Chief Data Office guides domain solutions, defines engineering standards, and drives data products that generate actionable insights across the organisation.

You will mentor teams, collaborate with product owners and stakeholders, and ensure delivery aligns with regulatory requirements and enterprise data governance.

Qualifications

• Degree in a technical discipline or equivalent practical experience.
• Proven senior/lead data engineer experience in complex enterprise environments.
• Strong hands-on data platforms and pipelines with Databricks on Azure.
• Experience with batch and real-time processing, event-driven architectures.
• Ability to define standards, lead design delivery, and mentor engineers.
• Knowledge of data governance, data lineage, metadata management, and security.

Responsibilities

• Lead the technical delivery of major data initiatives and ensure high-quality outcomes.
• Translate business requirements into detailed technical designs and roadmaps.
• Define and evolve data engineering standards, patterns, and best practices.
• Mentor engineers, conduct code reviews, and promote engineering excellence.
• Collaborate with product, analytics, architecture, and governance teams to align with enterprise standards.

Skills

Databricks

Azure

Python

SQL

Data modeling

Data governance

Observability

CI/CD

Quality engineering

Leadership

Education

Degree in a technical discipline (e.g. computer science, engineering, maths, physics) or equivalent practical experience

Tools

Databricks

Azure Data Factory

Delta Lake

Unity Catalog

Spark

Fabric

Git & CI/CD tools

Infra as code

Job description

Quilter plc leads data engineering with a strong focus on scalable, secure data platforms. The Lead Data Engineer within the Chief Data Office guides domain solutions, defines engineering standards, and drives data products that generate actionable insights across the organisation.

You will mentor teams, collaborate with product owners and stakeholders, and ensure delivery aligns with regulatory requirements and enterprise data governance.