Databricks Platform Engineer

Sagacity Solutions Ltd.

Greater London

Hybrid

GBP 85,000 - 120,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Hollybank Trustees Ltd in London is seeking a Platform Architect to design and implement scalable Databricks Lakehouse platforms on AWS and/or Azure, aligned to client requirements.

You will lead architecture sessions, run proofs of concept, and enable client teams through structured knowledge transfer while ensuring governance, security, and cost management across deployments.

Qualifications

  • 3+ years experience in data platform engineering or cloud engineering.
  • Hands-on with Databricks, Apache Spark, Delta Lake and Workflows.
  • Experience building data platforms on AWS and/or Azure (data lake, VNet/VPC, IAM).
  • Proficient with infrastructure-as-code (Terraform) and CI/CD (Azure DevOps, GitHub Actions).
  • Strong communication and client-facing consultancy skills.
  • Willingness to travel within the UK and right to work in the UK.

Responsibilities

  • Design and implement scalable Databricks Lakehouse platforms on AWS and/or Azure.
  • Architect end-to-end data platforms including ingestion, storage (Delta Lake), processing, and consumption layers.
  • Define environment strategies and CI/CD pipelines for Databricks deployments.
  • Produce architecture diagrams, ADRs, runbooks and deployment guides.
  • Enable client teams through knowledge transfer and training.
  • Collaborate with data engineers, data scientists and product teams.
  • Ensure governance, security and cost management across deployments.

Skills

Databricks
Apache Spark
Delta Lake
Workflows
Terraform
Azure DevOps
GitHub Actions
Security governance
RBAC/ABAC
CI/CD

Tools

Databricks
Delta Live Tables
Auto Loader
Terraform
Azure DevOps
GitHub Actions

Job description

Location: Hybrid / Monument St, London EC3R 8AJ, UK
job type: Permanent / Full-time
Sector and subsector: IT | Data Engineer
Salary: Competitive salary

Platform Architecture & Engineering responsibilities:
  • Design and implement scalable Databricks Lakehouse platforms on AWS and/or Azure aligned to client requirements
  • Architect end-to-end data platforms including ingestion, storage (Delta Lake), processing, and consumption layers
  • Build and configure cloud infrastructure using infrastructure-as-code (e.g. Terraform & Declarative Automation Bundles(DAB's))
  • Establish secure, compliant environments including networking (VNet/VPC, Private Link), identity (IAM/Entra ID), data governance (Unity Catalog), and access controls
  • Define environment strategies (dev/test/prod), CI/CD pipelines, and release processes for Databricks deployments
  • Implement monitoring, logging, cost optimisation, and performance tuning across the platform
  • Design and implement data pipelines using Delta Live Tables, Auto Loader, and Databricks Workflows for both batch and streaming workloads
Client Delivery & Enablement responsibilities:
  • Work directly with clients to translate business and technical requirements into scalable platform designs
  • Lead technical workshops, architecture sessions, and whiteboarding engagements with client stakeholders
  • Support rapid prototyping and proof-of-concept builds within Databricks to demonstrate platform capabilities and accelerate client adoption
  • Provide best practice guidance on Lakehouse architecture, data modelling, workload optimisation, and cost management
  • Produce high-quality technical documentation including architecture diagrams, architecture decision records (ADRs), runbooks, and deployment guides
  • Enable client teams through structured knowledge transfer, training, and platform handover
  • Collaborate with data engineers, data scientists, and product teams to ensure successful delivery outcomes
Governance & Security:
  • Implement Unity Catalog for centralised data governance, including access control (RBAC/ABAC), data lineage, audit logging, and compliance enforcement
  • Apply security best practices across platform design: network isolation, secret management, encryption at rest and in transit, and identity federation
  • Ensure platform designs meet client regulatory and compliance requirements (e.g. GDPR, ISO 27001, sector-specific standards)
What success looks like in the role:
  • Delivery of robust, secure, and scalable Databricks platforms that meet client performance and cost expectations
  • Clear, well-architected solutions that balance flexibility, governance, and operational efficiency
  • Strong client relationships built on trust, technical credibility, and effective communication
  • Accelerated client adoption of the Lakehouse platform through well-designed enablement and documentation
  • Reduced deployment time through reusable infrastructure patterns and automation
  • Proactive identification of risks, trade-offs, and optimisation opportunities across platform design and delivery
  • Contribution to the organisation’s growing body of reusable platform accelerators, reference architectures, and internal knowledge
Competencies and Behaviours:
  • 3+ years experience in data platform engineering, cloud engineering, or similar roles
  • Strong hands-on experience with Databricks, including Apache Spark, Delta Lake, Workflows
  • Proven experience designing and deploying data platforms on AWS and/or Azure (e.g. ADLS, S3, VNet/VPC, IAM)
  • Experience with infrastructure-as-code tools (e.g. Terraform preferred) and CI/CD pipelines (e.g. Azure DevOps, GitHub Actions)
  • Solid understanding of data architecture concepts including Lakehouse medallion architecture and dimensional modelling
  • Familiarity with security and governance frameworks (e.g. RBAC, ABAC, data masking, audit, compliance standards)
  • Excellent communication skills with the ability to explain complex technical concepts to non-technical stakeholders
  • Comfortable working in a client-facing consultancy environment with multiple concurrent engagements
  • Proactive, self-driven, and able to take ownership of end-to-end platform delivery
  • Willingness to travel within the UK as required
  • Right to work in the UK
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Databricks Lakehouse Platform Engineer (Hybrid, UK)
Databricks Lakehouse Platform Engineer (Hybrid, UK)

Sagacity Solutions Ltd. • Greater London

Hybrid
GBP 85,000 - 120,000
Databricks Engineer
Databricks Engineer

Gazelle Global Consulting Limited • Greater London

Hybrid
GBP 70,000 - 95,000
Databricks Architect
Databricks Architect

Project Recruit • Greater London

Hybrid
GBP 111,000 - 166,000
Databricks Engineer
Databricks Engineer

Gazelle Global • City Of London

Hybrid
GBP 85,000 - 120,000
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

ISx4 Group • Belfast City District

Hybrid
GBP 70,000 - 100,000
Databricks Data Engineer
Databricks Data Engineer

GIS Recruiter Ltd • Greater London

On-site
GBP 90,000 - 120,000
Databricks Solution Architect
Databricks Solution Architect

GIS Recruiter Ltd • City Of London

On-site
GBP 120,000 - 165,000
Databricks Architect
Databricks Architect

Gazelle Global Consulting Limited • Manchester

Hybrid
GBP 111,000 - 166,000
Databricks Data Engineer
Databricks Data Engineer

GIS Recruiter Ltd • City Of London

On-site
GBP 90,000 - 130,000
Databricks Architect
Databricks Architect

DCV Technologies • Greater London

On-site
GBP 90,000 - 140,000