Senior Databricks Architect & Data Transformation Leader

EY UK; City of Edinburgh; Hybrid; GBP 120,000 - 170,000; Full time

Benefits offered by this job

Flexible working

Career development

Total Rewards package

Job summary

EY UK is seeking a Senior Manager – Data Engineering to lead our Databricks architecture practice in London/Edinburgh. You will drive data transformation initiatives for financial services, shaping scalable data platforms and governance frameworks.

You will mentor teams, engage senior stakeholders, and ensure delivery of high-impact data products while advancing modern data technologies across the organization.

Qualifications

• Demonstrated track record in delivering large data transformations in FS sectors.
• Strong leadership and interpersonal skills with senior stakeholders.
• Experience building and leading high-performing data teams.
• Understanding of cloud-based data architectures and modern data technologies.

Responsibilities

• Lead design and implementation of scalable data architectures using Databricks on Azure.
• Architect and optimize data pipelines for structured and unstructured FS data.
• Stand up and operationalize Databricks platforms including workspace setup and CI/CD.
• Lead large-scale lift-and-shift migrations to Databricks with minimal disruption.
• Collaborate with cross-functional teams to align data architecture with regulations (GDPR, Basel III, MiFID II).
• Drive adoption of Delta Lake, Spark, and MLflow for analytics and ML use cases.
• Mentor data engineers and analysts across risk, trading, and compliance domains.
• Establish data governance, lineage, and metadata practices within Databricks.

Skills

Leadership

Stakeholder mgmt

Communication

Problem solving

Education

Bachelor’s degree or equivalent

Tools

Databricks

Azure

Delta Lake

Apache Spark

MLflow

Job description

EY UK is seeking a Senior Manager – Data Engineering to lead our Databricks architecture practice in London/Edinburgh. You will drive data transformation initiatives for financial services, shaping scalable data platforms and governance frameworks.

You will mentor teams, engage senior stakeholders, and ensure delivery of high-impact data products while advancing modern data technologies across the organization.