Databricks Architect

Project Recruit

Greater London

Hybrid

GBP 111,000 - 166,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Project Global Limited seeks a Databricks Architect to lead enterprise Databricks Lakehouse platforms for banking/financial services clients. Location is London or Manchester, UK, with hybrid working and a 3-days/week office presence. The role is a 4+ month contract starting ASAP, with a day rate and remote UK options.

You will build, secure, automate, and run platforms, covering workspace provisioning, IaC, CI/CD, security hardening, and multi-environment deployments across regions.

Qualifications

  • Extensive hands-on Databricks and Lakehouse experience.
  • Strong PySpark/SQL skills for transformations and data modeling.
  • Experience with Unity Catalog, Autoloader, and structured streaming.
  • Terraform or IaC for Databricks platform config, optional but beneficial.
  • Experience migrating workloads to Databricks and building batch/incremental pipelines.

Responsibilities

  • Build and maintain Databricks Lakehouse platforms across Dev/Test/UAT/Prod environments.
  • Provision and manage Databricks workspaces, compute resources, and policies.
  • Automate IaC for Databricks configuration and CI/CD pipelines.
  • Configure RBAC, secrets, and least-privilege access for secure ops.
  • Integrate Databricks with AWS/Azure cloud services and storage.

Skills

Databricks Notebooks
PySpark DataFrame API
SQL transformations
Delta Lake
Databricks Jobs/Workflows
Declarative Pipelines (dpx)
Unity Catalog
Databricks Autoloader/Structuredn
Terraform/IaC
Batch & incremental pipelines
AWS stack
S3
IAM
KMS
Glue/Lambda
Git/CI/CD
Lakehouse architecture
Data modeling (bronze-silver-gold)
Distributed data processing

Tools

Databricks
Terraform
IaC
AWS
Azure
S3
IAM
KMS
Glue
Lambda
Git

Job description

Databricks Architect

Our client, a leading global supplier for IT services, requires Databricks Architect to be based at their client’s office in London or Manchester, UK.

This is a hybrid role – you can work remotely in the UK and attend the London or Manchester office 3 days per week .

This is a 4+ month temporary contract to start asap

Day rate: Competitive Market rate

You will build, secure, automate, and run enterprise Databricks Lakehouse platforms for banking and financial services clients. The role covers workspace provisioning, infrastructure-as-code, CI/CD, security hardening, and platform operations across multiple environments and regions.

Key Responsibilities
  • Build, configure, and maintain enterprise Databricks Lakehouse platforms across Dev, Test, UAT, and Production environments.
  • Provision and manage Databricks workspaces, compute resources, cluster configurations, and platform-level settings.
  • Automate infrastructure provisioning and configuration using Terraform / Infrastructure as Code (IaC).
  • Develop and maintain CI/CD pipelines for Databricks platform configurations, notebooks, jobs, and application deployments.
  • Configure and administer Unity Catalog, including catalogs, schemas, permissions, and access controls.
  • Implement RBAC, service principals, secrets management, and least-privilege access for secure platform operations.
  • Integrate Databricks with cloud services such as AWS or Azure, including storage, networking, identity, and security services.
  • Configure and maintain cluster policies, job clusters, compute configurations, and environment standards.
  • Monitor platform health, workloads, jobs, and compute resources and troubleshoot production issues.
  • Implement logging, monitoring, alerting, and operational processes to improve platform reliability.
  • Optimize Databricks workloads and compute configurations for performance and cost efficiency.
  • Automate repetitive platform administration, deployment, and operational activities.
  • Support multi-environment and multi-region deployments, ensuring consistent platform configurations.
  • Work with Data Engineering, Cloud, DevOps, Security, Governance, and Application teams on platform integration.
  • Maintain technical documentation, deployment standards, operational procedures, and platform configurations.
  • Support security hardening and governance requirements appropriate for banking and financial-services environments.
Key Requirements
  • Databricks & Lakehouse
  • Strong hands-on experience with:
    • Databricks Notebooks
    • PySpark DataFrame API
    • SQL transformations
    • Delta Lake
    • Databricks Jobs, Job Flows, Workflows
    • Declarative Pipelines (dpx files, pipeline definitions)
  • Experience with Unity Catalog
  • Databricks Autoloader and Structured Streaming
  • Terraform or IaC experience for Databricks configuration (Optional)
  • Experience building both batch and incremental pipelines.
  • AWS Stack
  • Strong familiarity with:
    • S3, IAM, KMS
    • Glue / Lambda
  • Experience migrating workloads from AWS native tools to Databricks.
  • General Data Engineer
    • Distributed data processing and performance tuning.
    • Strong knowledge of Lakehouse medallion architecture
    • Data modeling for Lakehouse: bronze → silver → gold patterns.
    • Experience with Git, CI/CD.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Databricks Engineer
Databricks Engineer

Gazelle Global • City Of London

Hybrid
GBP 85,000 - 120,000
Databricks Solution Architect
Databricks Solution Architect

Undisclosed Company • Greater London

Hybrid
GBP 112,000 - 147,000
Data Architect
Data Architect

Vallum Associates • Greater London

On-site
GBP 90,000 - 130,000
Databricks Lakehouse Architect - Hybrid UK
Databricks Lakehouse Architect - Hybrid UK

Project Recruit • Greater London

Hybrid
GBP 111,000 - 166,000
Databricks Architect
Databricks Architect

DCV Technologies • City Of London

On-site
GBP 90,000 - 130,000
Databricks Platform Engineer
Databricks Platform Engineer

Sagacity Solutions Ltd. • Greater London

Hybrid
GBP 85,000 - 120,000
Databricks Data Architect
Databricks Data Architect

Gravitas Recruitment Group • City of Edinburgh, Manchester, Greater London

Hybrid
GBP 100,000 - 130,000
Data Architect
Data Architect

Brio Digital • United Kingdom

Remote
GBP 112,000 - 147,000
Resident Solution Architect
Resident Solution Architect

Technopride Ltd • Greater London

On-site
GBP 90,000 - 140,000
Databricks Lakehouse Architect | Data Platform Leader
Databricks Lakehouse Architect | Data Platform Leader

Undisclosed Company • Greater London

Hybrid
GBP 112,000 - 147,000