Azure Databricks Data Architect - AI-Ready Platform

Aztec Group; Greater London; Hybrid; GBP 90,000 - 120,000; Full time

Benefits offered by this job

Discretionary bonus scheme

Flexible, hybrid working

Ability to work abroad for up to 3 4?

Generous holiday allowance

Pension scheme

Private medical insurance

Life assurance (death in service and/…

Worldwide travel insurance

Health and wellbeing programmes

On-site parking

Job summary

Aztec Group in London is seeking an Enterprise Data Architect to design, implement and maintain data models for our Azure Data Platform, with a focus on the Databricks Lakehouse and Unity Catalog.

You will drive data governance, MD​M patterns, cost optimisation, and AI-ready data pipelines, collaborating with AI/ML teams and external clients while ensuring GDPR compliance and secure data exchange.

Qualifications

• Deep, hands-on proficiency with Databricks, including Unity Catalog, Delta Lake, workflows, and platform administration concepts.
• Practical experience with Databricks Delta Sharing and designing integration patterns for external data sharing, including the security and contractual considerations involved.
• Strong track record designing and implementing Master Data Management solutions, including tooling selection, data stewardship models, and reference data management.
• Demonstrable expertise in Databricks performance and cost optimisation at scale.
• Strong data modelling skills (dimensional, 3NF, Data Vault or similar) and experience defining enterprise data models.
• Solid grounding in Azure, including storage, networking, and identity concepts relevant to data platforms.
• Experience with data governance frameworks and tools (e.g. Microsoft Purview), data quality management, metadata, and lineage tooling.
• Strong SQL and working knowledge of Python and/or Spark.
• Understanding of data security and privacy: encryption, access control (RBAC/ABAC), data masking, and regulatory compliance (GDPR).
• Excellent communication skills, with the ability to explain architectural decisions in plain language to executive and non-technical audiences.

Responsibilities

• Define and maintain the enterprise data architecture, including target-state designs, roadmaps, and architectural standards and patterns.
• Own the architecture of our Databricks Lakehouse platform, including workspace design, Unity Catalog structure, medallion architecture, and environment strategy.
• Design and govern integration patterns for sharing data with external clients and third parties, including secure, scalable approaches using Databricks Delta Sharing, APIs, and file-based exchange where appropriate.
• Lead the design and adoption of Master Data Management (MDM), including entity definitions, golden-record strategy, match-and-merge rules, and stewardship processes.
• Establish standards and practices for optimising Databricks performance and cost — cluster and compute strategy, Delta table, caching, and workload right-sizing.
• Define data modelling standards (conceptual, logical, physical) across analytical and operational domains.
• Work with data governance colleagues to embed data quality, lineage, cataloguing, classification, and access-control policies into platform design.
• Ensure architectures meet security, privacy, and regulatory requirements, including GDPR and data residency considerations.
• Review solution designs, provide architectural assurance, and support engineering teams through design authority and peer review.
• Design data architectures that enable AI adoption — ensuring data is discoverable, well-governed, and AI-ready for machine learning, GenAI, and agentic use cases across the business.
• Partner with AI/ML teams to define patterns for feature engineering, model data pipelines, vector search, and the safe exposure of enterprise data to AI systems.
• Evaluate emerging technologies and make pragmatic build/buy/adopt recommendations, including data and AI enablement opportunities.

Skills

Databricks

Unity Catalog

Delta Lake

Workflows

Platform administration

Delta Sharing

Master Data Management

Data modelling

Azure

Purview

SQL

Python

Spark

RBAC/ABAC

GDPR

Communication

Tools

Microsoft Purview

Azure storage

Azure networking

Job description

Aztec Group in London is seeking an Enterprise Data Architect to design, implement and maintain data models for our Azure Data Platform, with a focus on the Databricks Lakehouse and Unity Catalog.

You will drive data governance, MD​M patterns, cost optimisation, and AI-ready data pipelines, collaborating with AI/ML teams and external clients while ensuring GDPR compliance and secure data exchange.