Principal Data Architect - Databricks & AI

wk; Greater London; On-site; GBP 85,000 - 120,000; Full time

Job summary

wk is seeking a Principal Data Architect to define and own the data architecture for a strategic transformation. You will lead the design of a modern Databricks lakehouse and medallion patterns, enabling secure, scalable data use across a complex product landscape.

You will shape canonical and semantic data models, govern data pipelines, and drive privacy-preserving patterns for AI/analytics workloads, while mentoring engineers and aligning with product and security leaders.

Qualifications

• Significant experience as a Data Architect or similar senior technical leader.
• Advanced, hands-on Databricks experience in production environments.
• Proven ownership of modern data platforms from architecture to delivery.
• Strong experience in data privacy, security and governance patterns.
• Data-modelling expertise across canonical and dimensional models.
• Experience with batch and real-time data integration and APIs.

Responsibilities

• Define target data architecture, standards and technical roadmap.
• Design a Databricks lakehouse foundation for secure, scalable data use across products.
• Establish medallion patterns (Bronze/Silver/Gold) for data pipelines.
• Design canonical, dimensional and semantic data models for interoperability.
• Architect data integration using batch, streaming, APIs and CDC.
• Design privacy-preserving data capabilities for AI workloads.
• Create prototypes, review designs, and guide performance optimization.
• Build production-ready solutions with governance, observability and cost control.
• Collaborate with product, engineering, security and business leaders to translate needs into plans.
• Mentor data engineers and promote reusable platform capabilities.

Skills

Databricks lakehouse

Python/SQL/Scala/Java

Azure data services

Data modelling

Data governance

Security/privacy

Architectural leadership

Mentoring engineers

Tools

dbt

Kafka

Airflow

Terraform

Power BI

Tableau

Collibra/Microsoft Purview

Job description

The opportunity

We are investing in the next generation of our data and technology capabilities to create more connected, intelligent experiences for customers. Data is central to this ambition. This is an opportunity to define the architecture, standards and delivery patterns for a modern data platform that will enable trusted data use across products and support advanced analytics and AI.

The Principal Data Architect will be the senior technical authority for data architecture within this strategic transformation. The role is architecture-led while remaining practically engaged through prototyping, proof-of-concepts, technical reviews and the resolution of complex engineering challenges. As the capability grows, the remit may expand to include leadership of a small team .

What you will do

• Define and own the target data architecture, standards, reference patterns and technical roadmap for a strategic enterprise data capability .
• Design a modern Databricks lakehouse foundation that enables secure, scalable use of data across a complex product landscape .
• Establish scalable medallion patterns across Bronze, Silver and Gold layers, including ingestion, transformation, storage, serving and consumption.
• Design canonical , dimensional and semantic data models that enable consistent exchange, interoperability and reuse across product domains.
• Architect secure data integration using batch, streaming, APIs, change data capture and event-driven patterns.
• Design privacy-preserving data capabilities, including data masking, anonymisation or equivalent controls for customer data used in AI and analytical workloads.
• Design AI-ready data capabilities, including governed model-data pipelines, vector search and retrieval-augmented generation patterns where relevant.
• Create prototypes and proof-of-concepts, review technical designs and code, guide performance optimisation , and help resolve complex technical issues.
• Build production-ready solutions with strong governance, lineage, quality, observability, security, reliability and cost control.
• Partner with product, engineering, AI, security, platform and business leaders to translate customer and product needs into architecture and delivery plans.
• Mentor data engineers and architects and promote reusable platform capabilities and engineering standards.

What you bring

• Significant experience as a Data Architect, Lead Data Architect, Principal Data Engineer or comparable senior technical leader.
• Advanced, hands-on Databricks experience in production environments, including lakehouse architecture, Delta Lake and relevant governance, workflow, SQL and optimisation capabilities.
• Proven ownership of modern data platforms or data products from architecture through production delivery and operation.
• Strong experience in data privacy and security, including data masking, anonymisation , tokenisation or comparable privacy-preserving patterns for sensitive or customer data.
• Strong data-modelling expertise across conceptual, logical, physical, dimensional and canonical models.
• Experience with batch and real-time integration, including APIs, CDC, streaming or event-driven pipelines.
• A strong data-engineering or software-development foundation using Python, SQL, Scala, Java and/or Spark.
• Deep experience with Azure data services. AWS experience is advantageous as the platform expands.
• Experience with governance, cataloguing, lineage, quality, metadata, access control and secure multi-tenant or customer-data environments.
• Credibility with hands-on engineers and senior stakeholders, with strong judgement across speed, scale, governance, cost and usability.

Valuable additional experience

• Production AI/ML or generative-AI data architecture, including RAG, embeddings, vector search or model-data pipelines.
• Knowledge graphs, ontologies, RDF, linked data or advanced semantic technologies.
• Databricks or cloud architecture/ data-engineering certification.
• Tools such as dbt , Kafka, Airflow, Terraform, Power BI, Tableau, Collibra or Microsoft Purview.
• Architecture across multiple products, domains or business units in B2B software, information services, consulting or regulated environments.

Why join

This is a rare opportunity to shape a strategically important data foundation from the beginning. You will have the mandate to make high-impact architectural decisions, work on complex technical challenges and influence how trusted data powers better customer experiences, advanced analytics and AI. The role offers meaningful visibility, broad collaboration and the chance to create capabilities that will scale across an evolving product landscape.

Our Interview Practices

To maintain a fair and genuine hiring process, we kindly ask that all candidates participate in interviews without the assistance of AI tools or external prompts. Our interview process is designed to assess your individual skills, experiences, and communication style. We value authenticity and want to en