Databricks Architect

DCV Technologies

City Of London

Vor Ort

GBP 90.000 - 130.000

Vollzeit

Vor 2 Tagen
Sei unter den ersten Bewerbenden

Erhalte eine Antwort von diesem Arbeitgeber

Dein Lebenslauf und dein Anschreiben, genau auf das zugeschnitten, was gesucht wird.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

DCV Technologies is seeking a skilled Databricks Platform Engineer to build, secure, automate, and run enterprise Databricks Lakehouse platforms for banking and financial services clients across multiple environments. You will provision workspaces, implement IaC with Terraform, develop CI/CD for notebooks and jobs, configure Unity Catalog, RBAC, and integrate with AWS/Azure.

This role requires collaboration with Data Engineering, Cloud, DevOps, Security, and Governance teams to meet stringent

Qualifikationen

  • Hands-on Databricks Lakehouse experience.
  • Experience with Unity Catalog and access control.
  • Strong knowledge of AWS services and Spark SQL.

Aufgaben

  • Build and secure enterprise Databricks Lakehouse platforms.
  • Provision Databricks workspaces and resources across environments.
  • Automate infrastructure with Terraform / IaC.
  • Develop CI/CD pipelines for notebooks, jobs and configurations.
  • Configure Unity Catalog, RBAC, and secrets management.
  • Integrate Databricks with AWS/Azure services.
  • Monitor health, optimize workloads, and ensure security governance.
  • Collaborate with Data Engineering, Cloud, DevOps, Security, Governance teams.

Kenntnisse

Databricks Notebooks
PySpark DataFrame
SQL transformations
Delta Lake
Databricks Jobs
Databricks Workflows
Declarative Pipelines
Unity Catalog
AWS Stack
S3
IAM
KMS
Terraform / IaC
CI/CD

Tools

Terraform
IaC

Jobbeschreibung

The Role

You will build, secure, automate, and run enterprise Databricks Lakehouse platforms for banking and financial services clients. The role covers workspace provisioning, infrastructure-as-code, CI/CD, security hardening, and platform operations across multiple environments and regions.

Your responsibilities
  • Build, configure, and maintain enterprise Databricks Lakehouse platforms across Dev, Test, UAT, and Production environments.
  • Provision and manage Databricks workspaces, compute resources, cluster configurations, and platform-level settings.
  • Automate infrastructure provisioning and configuration using Terraform / Infrastructure as Code (IaC).
  • Develop and maintain CI/CD pipelines for Databricks platform configurations, notebooks, jobs, and application deployments.
  • Configure and administer Unity Catalog, including catalogs, schemas, permissions, and access controls.
  • Implement RBAC, service principals, secrets management, and least-privilege access for secure platform operations.
  • Integrate Databricks with cloud services such as AWS or Azure, including storage, networking, identity, and security services.
  • Configure and maintain cluster policies, job clusters, compute configurations, and environment standards.
  • Monitor platform health, workloads, jobs, and compute resources and troubleshoot production issues.
  • Implement logging, monitoring, alerting, and operational processes to improve platform reliability.
  • Optimize Databricks workloads and compute configurations for performance and cost efficiency.
  • Automate repetitive platform administration, deployment, and operational activities.
  • Support multi-environment and multi-region deployments, ensuring consistent platform configurations.
  • Work with Data Engineering, Cloud, DevOps, Security, Governance, and Application teams on platform integration.
  • Maintain technical documentation, deployment standards, operational procedures, and platform configurations.
  • Support security hardening and governance requirements appropriate for banking and financial-services environments.
Your Profile
Essential skills/knowledge/experience:
Databricks & Lakehouse
  • Strong hands-on experience with:
    • Databricks Notebooks
    • PySpark DataFrame API
    • SQL transformations
    • Delta Lake
    • Databricks Jobs, Job Flows, Workflows
    • Declarative Pipelines (dpx files, pipeline definitions)
Experience with Unity Catalog
Databricks Autoloader and Structured Streaming
Terraform or IaC experience for Databricks configuration (Optional)

Experience building both batch and incremental pipelines.

AWS Stack
Strong familiarity with:
  • S3, IAM, KMS
  • Glue / Lambda

Experience migrating workloads from AWS native tools to Databricks.

General Data Engineering
  • Distributed data processing and performance tuning.
  • Strong knowledge of Lakehouse medallion architecture
  • Data modeling for Lakehouse: bronze → silver → gold patterns.
  • Experience with Git, CI/CD
Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

Data Architect
Data Architect

Vallum Associates • Greater London

Vor Ort
GBP 90.000 - 130.000
Databricks Architect
Databricks Architect

Project Recruit • Greater London

Hybrid
GBP 111.000 - 166.000
Senior Databricks Solution Architect
Senior Databricks Solution Architect

Unison Group • Norwich

Vor Ort
GBP 120.000 - 160.000
Databricks Platform Engineer
Databricks Platform Engineer

Sagacity Solutions Ltd. • Greater London

Hybrid
GBP 85.000 - 120.000
Databricks Engineer
Databricks Engineer

Gazelle Global • City Of London

Hybrid
GBP 85.000 - 120.000
Senior Enterprise Databricks Lakehouse Architect
Senior Enterprise Databricks Lakehouse Architect

DCV Technologies • City Of London

Vor Ort
GBP 90.000 - 130.000
Staff Data Engineer
Staff Data Engineer

numi • Greater London

Vor Ort
GBP 85.000 - 120.000
Enterprise Databricks Lakehouse Architect
Enterprise Databricks Lakehouse Architect

Vallum Associates • Greater London

Vor Ort
GBP 90.000 - 130.000
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

ISx4 Group • Belfast City District

Hybrid
GBP 70.000 - 100.000
Databricks Data Engineer (Contract)
Databricks Data Engineer (Contract)

Swift Analytics • Großbritannien

Remote
GBP 70.000 - 90.000