Senior Databricks Architect & Data Engineering Leader

EY; Greater London; On-site; GBP 120,000 - 170,000; Full time

Benefits offered by this job

Career progression

Flexible working

Job summary

EY Financial Services invites applications for a Senior Manager - Data Engineering focusing on Databricks architecture in London/Edinburgh. You will lead scalable data solutions, optimize pipelines and modern data platforms to support risk, trading and compliance domains.

You will mentor teams, advance data governance and partner with senior stakeholders to deliver high-impact outcomes in regulatory and business contexts.

Qualifications

• A track record of delivering large Databricks data transformations and appropriate Databricks industry certifications.
• Deep understanding of Financial Services to develop AI, Data and Analytics solutions.
• Experience building high-performing teams and exceptional leadership and interpersonal skills.
• Understanding of cloud-based technologies and data platforms with market trends.

Responsibilities

• Lead the design and implementation of scalable data architecture solutions using Databricks on Azure.
• Architect and optimise data pipelines for structured and unstructured financial data.
• Demonstrate experience in standing up and operationalising Databricks platforms, including workspace setup and CI/CD.
• Lead large-scale lift-and-shift data migrations with minimal disruption and high data fidelity.
• Collaborate with cross-functional teams to align data architecture with financial regulations (GDPR, Basel III, MiFID II).
• Drive adoption of Delta Lake, Apache Spark and MLflow for analytics and ML use cases.
• Provide technical leadership and mentoring to data engineers and analysts across financial domains.
• Establish best practices for data governance, lineage and metadata management in Databricks.

Skills

Databricks

Azure

Data pipelines

CI/CD

Delta Lake

Apache Spark

MLflow

Data governance

Security hardening

Leadership

Stakeholder management

Education

University degree (2:1 or above)

Job description

EY Financial Services invites applications for a Senior Manager - Data Engineering focusing on Databricks architecture in London/Edinburgh. You will lead scalable data solutions, optimize pipelines and modern data platforms to support risk, trading and compliance domains.

You will mentor teams, advance data governance and partner with senior stakeholders to deliver high-impact outcomes in regulatory and business contexts.