Senior Manager, Databricks Architect, AI&Data, FS

Ernst & Young Advisory Services Sdn Bhd; Greater London; On-site; GBP 120,000 - 180,000; Full time

Benefits offered by this job

Flexible working

Total Rewards package

Career development

Job summary

EY is seeking a Senior Manager to lead on Databricks Architectures within the AI & Data practice in London, focusing on data engineering capabilities and compliances for the Financial Services sector.

The role emphasizes delivering data transformation programs, collaborating with cross-functional teams, and guiding data governance and security in cloud environments.

Qualifications

• Experience delivering large Databricks data transformations; Databricks certifications preferred.
• Deep understanding of Financial Services to tailor AI/Data/Analytics solutions.
• Proven ability to build high-performing teams and lead others.
• Strong understanding of cloud technologies and modern data tech trends.
• Ability to deliver across multiple engagements; manage quality and risk.
• Ability to engage with senior stakeholders on large cross-functional projects.
• Excellent problem-solving, storytelling and business writing skills.
• University degree 2:1 or equivalent preferred; open to outstanding alternatives.

Responsibilities

• Lead scalable data architecture design and implementation using Databricks on Azure.
• Architect and optimize data pipelines for structured and unstructured financial data.
• Demonstrate experience standing up and operationalizing Databricks platforms (workspace, clusters, CI/CD).
• Lead large-scale lift-and-shift migrations to Databricks with minimal disruption and high fidelity.
• Collaborate with cross-functional teams to align data architecture with financial regulations (GDPR, Basel III, MiFID II).
• Drive adoption of Delta Lake, Apache Spark and MLflow for analytics and ML use cases.
• Provide technical leadership and mentoring to data engineers and analysts across financial domains.
• Establish best practices for data governance, lineage and metadata management within Databricks.
• Translate business needs into robust data products that enable real-time decision making and predictive modeling.
• Lead performance tuning, cost optimization, and security hardening of Databricks on Azure.

Skills

Databricks on Azure

Data pipelines

Databricks platform operation

Lift-and-shift migrations

Financial regulations (GDPR, Basel III

Delta Lake

Apache Spark

MLflow

Data governance

Data products

Security hardening

Leadership & mentoring

Education

University degree, 2:1 (or equivalent)

Tools

Databricks on Azure

CI/CD integration

Delta Lake

Apache Spark

MLflow

Job description

Select how often (in days) to receive an alert:

Select how often (in days) to receive an alert:

Senior Manager, Databricks Architect, AI&Data, FS

Location: London

Other locations: Primary Location Only

Date: 9 Aug 2026

Requisition ID: 1639394

At EY, we’re all in to shape your future with confidence.

We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go.

Join EY and help to build a better working world.

Senior Manager – Data Engineering – Databricks Architect

London/ Edinburgh

The opportunity

The EY Financial Services AI & Data team is continuing to grow and develop unique advisory propositions aligned to market needs and demand. We are currently looking for an exceptional Senior Manager to lead on developing our Data Engineering Capability with an initial focus on Databricks. This is a senior architecture role and will require an individual with an exceptional track record of delivering data transformation programs in the Financial Services sector.

The AI & Data practice works collaboratively with our clients to enhance their ability to solve complex business problems by exploiting data and analytics strategies and solutions. We provide expertise and delivery in three core areas which all work closely together:

• Data Architecture & Engineering – Focusing on next generation data solution architecture design and implementation
• Data Management & Strategy – Supporting our clients with Data Strategy, Data Governance Data and Analytics Target Operating Model Design and wider Data Office and CDO support functions
• Data Science and Information Analysis - Working with our clients to design build and integrate Machine Learning and AI solutions into their core business platforms and processes to drive greater insight and increased process efficiency and automation. We also help them design and implement modern data visualisation and reporting solutions

Your key responsibilities

• Lead the design and implementation of scalable data architecture solutions usingDatabricks on Azure.
• Architect and optimizedata pipelinesfor structured and unstructured financial data, ensuring performance, reliability, and compliance.
• Demonstrateprior experience in standing up and operationalizing Databricks platforms, including workspace setup, cluster configuration, and CI/CD integration.
• Lead and executelarge-scale lift-and-shift data migrationsfrom legacy platforms to Databricks, ensuring minimal disruption and high data fidelity.
• Collaborate with cross-functional teams to align data architecture withfinancial regulations(e.g., GDPR, Basel III, MiFID II).
• Drive adoption ofDelta Lake,Apache Spark, andMLflowfor advanced analytics and machine learning use cases.
• Provide technical leadership and mentoring to data engineers and analysts across financial domains such as risk, trading, and compliance.
• Establish best practices fordata governance, lineage, and metadata management within the Databricks environment.
• Translate business needs into robustdata productsthat support real-time decision-making and predictive modeling.
• Lead performance tuning, cost optimization, andsecurity hardeningof Databricks on Azure.

To qualify for the role, you must have

• A track record of delivering large Databricks data transformations & appropriate Databricks industry certifications
• A deep understanding of Financial Services to develop the right AI, Data and Analytics solutions to meet their needs
• Experience of building high performing teams and exceptional leadership and interpersonal skills
• Deep understanding of the emerging technology trends including adoption of cloud-based technologies and advance data technologies with a clear perspective on where the market is going.
• Ability to successfully deliver across multiple engagements, exceed client expectations, manage commercial aspects and manage quality and risk
• Ability to engage with senior stakeholders and manage large cross functional projects
• Intellectual strength / flexibility to rapidly understand complex problems and rationalise these into workable solutions which can then be delivered
• Exceptional storytelling, communication, presentation, business writing and professional skills, including being able to liaise with both technical and business contacts at our clients.
• Previous consulting experience will be greatly valued
• A university degree, 2:1 (or equivalent) or above is usually preferred, however we would be open to outstanding candidates who may have followed an alternative academic path.

What we look for

We are looking for highly motivated individuals who are passionate about AI, Data and Analytics and want to assist clients to become truly data driven organisations. If you think you can meet the challenges of a focused consulting organisation, understand how to grow and lead in a large practice and work in the most complex of sectors our AI & Data team is looking forward to hearing from you.

What working at EY offers

We offer a competitive remuneration package where you’ll be rewarded for your individual and team performance. Our comprehensive Total Rewards package includes support for flexible working and career development, and with FlexEY you can select benefits that suit your needs, covering holidays, health and well-being, insurance, savings and a wide range of discounts, offers and promotions. Plus, we offer:

• Support, coaching and feedback from some of the most engaging colleagues around
• Opportunities to develop new skills and progress your career
• The freedom and flexibility to handle your role in a way that’s right for you

About EY

As a global leader in assurance, tax, transaction and advisory services, we’re using the finance products, expertise and systems we’ve developed to build a better working world. That starts with a culture that believes in giving you the training, opportunities and creative freedom to make things better. Whenever you join, however long you stay, the exceptional EY experience lasts a lifetime. And with a commitment to hiring and developing the most passionate people, we’ll make our ambition to be the best employer by 2020 a reality.

Join us in building a better working world

Who we are

EY is committed to being an inclusive employer and we are happy to consider flexible working arrangements. We strive to achieve the right balance for our people, enabling us to deliver excellent client service whilst allowing you to build your career without sacrificing your personal priorities. While our client-facing professionals can be required to travel regularly, and at times be based at client sites, our flexible working arrangements can help you to achieve a lifestyle balance

Prior to finalizing your application, you will be asked to provide personal information across several dimensions of diversity and inclusiveness. The information you provide is kept entirely confidential and will not be used to evaluate your candidacy. We collect this data to help us analyse our recruitment process holistically and implement actions that promote diversity and inclusiveness. While optional, we encourage you to provide this information to hold us accountable towards our goal of building a better working world. We ask because it matters!

EY | Building a better working world

EY is building a better working world by creating new value for clients, people, society and the planet, while building trust in capital markets.

Enabled by data, AI and advanced technology, EY teams help clients shape the future with confidence and develop answers for the most pressing issues of today and tomorrow.

EY teams work across a full spectrum of services in assurance, consulting, tax, strategy and transactions. Fueled by sector insights, a globally connected, multi-disciplinary network and diverse ecosystem partners, EY teams can provide services in more than 150 countries and territories.

EY refers to the global organization, and may refer to one or more, of the member firms of Ernst & Young Global Limited, each of which is a separate legal entity. Ernst & Young Global Limited, a UK company limited by guarantee, does not provide services to clients.