Platform Architect

HelloKindred; Greater London; Hybrid; GBP 90,000 - 150,000; Full time

Job summary

HelloKindred in London is seeking a Platform Architect to optimise an enterprise Data Platform using Azure Databricks. You will enhance serverless capabilities, drive FinOps practices, and strengthen governance while modernising analytics workloads.

The role focuses on migration of POSIT/RStudio workloads, cost attribution and secure integrations, with hybrid work arrangements and a strong emphasis on scalable data ingestion and CI/CD-driven delivery.

Qualifications

• Bachelor's degree or equivalent in Computer Science, Engineering or related field.
• Proven experience with Azure Databricks and cloud data platforms.
• Strong knowledge of PySpark, Python, SQL and Delta Lake.
• Experience in FinOps, cost governance and platform operations.
• Familiarity with CI/CD for data platforms and secure data ingestion.

Responsibilities

• Assess workloads and select compute models (Serverless, Jobs, Interactive, Classic Clusters) based on SLA, performance and cost.
• Configure and implement Databricks Serverless capabilities across notebooks, jobs, SQL workloads, and data pipelines.
• Develop workload placement standards and identify cost-optimal strategies.
• Implement compute policies, autoscaling, quotas, budget controls and guardrails.
• Monitor platform performance and costs; identify oversized or idle resources.
• Define FinOps operating model covering ownership, environments, projects, and cost centres.
• Establish mandatory tagging and CI/CD validation for cost governance.
• Deliver granular cost attribution and reporting across workspaces and workloads.
• Configure budgets, alerts and usage monitoring to proactively manage platform costs.
• Analyse usage and billing data to identify anomalies and inefficiencies.
• Enhance the Databricks Discovery Zone to support migration of workloads.
• Enable secure API integrations, external data ingestion, BI connectivity and orchestration.
• Design reusable onboarding and migration patterns to reduce sprawl and improve security.
• Build scalable data ingestion and transformation pipelines using Python, PySpark, SQL and Delta Lake.

Skills

Databricks

Serverless

PySpark

Python

SQL

Delta Lake

FinOps

CI/CD

Security

APIs

Data Governance

Education

Bachelor's degree in Computer Science

Tools

Azure Databricks

Job description

Anticipated Contract End Date/Length: Approximately 4.5 months Work Set Up: Hybrid (60% office, 40% remote) Clearance Required: BPSS

Our client in the Information Technology and Services industry is looking for a Platform Architect to enhance and optimise an enterprise Data Platform through hands-on Azure Databricks architecture, engineering, operational support, and platform optimisation. The role will focus on enabling Databricks Serverless capabilities, strengthening FinOps practices and platform governance, and enhancing the Databricks Discovery Zone to support migration and consolidation of POSIT/RStudio workloads.

What you will do:

• Assess existing workloads and select appropriate compute models, including Serverless, Jobs Compute, Interactive Compute, and Classic Clusters, based on workload, SLA, performance, utilisation, and cost.
• Configure and implement Databricks Serverless capabilities across notebooks, jobs, SQL workloads, analytical processing, and data pipelines.
• Develop workload placement standards and guidance, identifying scenarios where Serverless may not provide the most cost-effective solution.
• Implement compute policies, autoscaling, quotas, budget controls, and operational guardrails.
• Monitor platform performance and costs, identifying oversized, underutilised, idle, or inefficient resources.
• Define and implement a practical FinOps operating model covering ownership, accountability, environments, projects, applications, cost centres, and teams.
• Establish mandatory tagging standards and integrate automated validation into CI/CD pipelines.
• Deliver granular cost attribution and reporting across workspaces, projects, applications, workloads, and business teams.
• Configure budgets, spend thresholds, alerts, and usage monitoring to proactively manage platform costs.
• Analyse usage and billing data to identify cost anomalies, inefficient workloads, excessive storage, unnecessary data movement, and underutilised resources.
• Enhance the Databricks Discovery Zone to support migration and modernisation of analytics and data science workloads currently running on POSIT/RStudio.
• Enable secure API and external system integrations, external data ingestion, BI and reporting connectivity, scheduling and orchestration, local IDE-based development, and LLM and AI integration.
• Define reusable onboarding, migration, and delivery patterns that reduce technology sprawl while improving platform security, supportability, and delivery speed.
• Design, build, and optimise scalable data ingestion and transformation solutions using Python, PySpark, SQL, and Delta Lake.
• Implement batch and incremental processing patterns, including CDC, schema evolution, reconciliation, data quality controls, and error handling.
• Develop reusable integration frameworks for REST APIs, SaaS platforms, databases, files, object storage, document repositories, enterprise systems, and external data sources.
• Implement secure authentication, secrets management, and credential handling practices.
• Deliver end-to-end data flows from source ingestion through governed and curated data layers supporting analytics, BI, machine learning, and application consumption.
• Support production platforms and drive continuous optimisation across architecture, engineering, implementation, and operational support.
• Document standards, patterns, operational procedures, and architectural decisions for technical and business stakeholders.