Databricks Platform Architect – FinOps & Serverless

HelloKindred; Greater London; Hybrid; GBP 90,000 - 150,000; Full time

Job summary

HelloKindred in London is seeking a Platform Architect to optimise an enterprise Data Platform using Azure Databricks. You will enhance serverless capabilities, drive FinOps practices, and strengthen governance while modernising analytics workloads.

The role focuses on migration of POSIT/RStudio workloads, cost attribution and secure integrations, with hybrid work arrangements and a strong emphasis on scalable data ingestion and CI/CD-driven delivery.

Qualifications

• Bachelor's degree or equivalent in Computer Science, Engineering or related field.
• Proven experience with Azure Databricks and cloud data platforms.
• Strong knowledge of PySpark, Python, SQL and Delta Lake.
• Experience in FinOps, cost governance and platform operations.
• Familiarity with CI/CD for data platforms and secure data ingestion.

Responsibilities

• Assess workloads and select compute models (Serverless, Jobs, Interactive, Classic Clusters) based on SLA, performance and cost.
• Configure and implement Databricks Serverless capabilities across notebooks, jobs, SQL workloads, and data pipelines.
• Develop workload placement standards and identify cost-optimal strategies.
• Implement compute policies, autoscaling, quotas, budget controls and guardrails.
• Monitor platform performance and costs; identify oversized or idle resources.
• Define FinOps operating model covering ownership, environments, projects, and cost centres.
• Establish mandatory tagging and CI/CD validation for cost governance.
• Deliver granular cost attribution and reporting across workspaces and workloads.
• Configure budgets, alerts and usage monitoring to proactively manage platform costs.
• Analyse usage and billing data to identify anomalies and inefficiencies.
• Enhance the Databricks Discovery Zone to support migration of workloads.
• Enable secure API integrations, external data ingestion, BI connectivity and orchestration.
• Design reusable onboarding and migration patterns to reduce sprawl and improve security.
• Build scalable data ingestion and transformation pipelines using Python, PySpark, SQL and Delta Lake.

Skills

Databricks

Serverless

PySpark

Python

SQL

Delta Lake

FinOps

CI/CD

Security

APIs

Data Governance

Education

Bachelor's degree in Computer Science

Tools

Azure Databricks

Job description

HelloKindred in London is seeking a Platform Architect to optimise an enterprise Data Platform using Azure Databricks. You will enhance serverless capabilities, drive FinOps practices, and strengthen governance while modernising analytics workloads.

The role focuses on migration of POSIT/RStudio workloads, cost attribution and secure integrations, with hybrid work arrangements and a strong emphasis on scalable data ingestion and CI/CD-driven delivery.