Senior Manager: Databricks Architect for AI & Data (FS)

EY UK; Greater London; On-site; GBP 120,000 - 180,000; Full time

Job summary

EY’s Financial Services AI & Data team seeks a Senior Manager to lead Databricks Data Engineering architecture in a FS context. You will drive scalable data solutions, mentor engineers, and partner with stakeholders to deliver compliant data transformations and real-time analytics.

The role emphasizes design leadership, cross-functional collaboration, and hands-on optimization of Databricks platforms to support risk, trading, and regulatory reporting across London and Edinburgh.

Qualifications

• Proven track record delivering large Databricks data transformations.
• Deep understanding of Financial Services to tailor AI, Data and Analytics.
• Experience building high performing teams and strong leadership.
• Comfort engaging with senior stakeholders and managing large projects.
• Knowledge of cloud and advanced data technologies shaping the market.
• Ability to deliver across multiple engagements with quality and risk focus.
• Degree holder with strong academic credentials or outstanding alternatives.

Responsibilities

• Lead design and implementation of scalable data architecture using Databricks on Azure.
• Architect and optimize data pipelines for structured and unstructured financial data, ensuring performance, reliability, and compliance.
• Demonstrate experience in standing up and operationalizing Databricks platforms including workspace setup and CI/CD.
• Lead and execute large-scale lift-and-shift data migrations from legacy platforms to Databricks.
• Collaborate with cross-functional teams to align data architecture with GDPR, Basel III, MiFID II.
• Drive adoption of Delta Lake, Apache Spark, and MLflow for analytics and ML use cases.
• Provide technical leadership and mentoring to data engineers and analysts across financial domains.
• Establish best practices for data governance, lineage, and metadata management in Databricks.
• Translate business needs into robust data products for real-time decision-making and predictive modeling.
• Lead performance tuning, cost optimization, and security hardening of Databricks on Azure.

Skills

Databricks data transformations

Leadership

Stakeholder engagement

Communication & storytelling

Cloud-based data technologies

Consulting experience

Data governance & compliance

Education

University degree (2:1 or equivalent)

Tools

Databricks on Azure

Delta Lake

Apache Spark

MLflow

Job description

EY’s Financial Services AI & Data team seeks a Senior Manager to lead Databricks Data Engineering architecture in a FS context. You will drive scalable data solutions, mentor engineers, and partner with stakeholders to deliver compliant data transformations and real-time analytics.

The role emphasizes design leadership, cross-functional collaboration, and hands-on optimization of Databricks platforms to support risk, trading, and regulatory reporting across London and Edinburgh.