Data Platform Architect - Databricks

eTeam Workforce Limited; Greater London; Hybrid; GBP 90,000 - 140,000; Full time

Job summary

eTeam Workforce Limited is seeking a Data Platform Architect to lead the evolution of our Enterprise Data Platform (EDP) with strong Databricks architecture and platform administration expertise. The role combines strategic architecture leadership with DevOps, FinOps, and data engineering to deliver business value and cost efficiencies.

The candidate will guide platform modernization, design Databricks Serverless capabilities, govern costs, and enable AI/ML integration.

Qualifications

• SFIA level 6 or above.
• Hands-on experience with Azure cloud services.
• Deep understanding of DevOps, CI/CD, Infrastructure as Code, and automation frameworks.
• Proven experience architecting large-scale cloud-based data platforms.
• Strong expertise in Databricks architecture and platform administration.
• Experience implementing FinOps practices and cloud cost optimization strategies.
• Strong understanding of data governance, security, and compliance requirements.
• Experience leading platform transformation and modernization programmes.
• Stakeholder management experience across technical and business functions.

Responsibilities

• Lead platform modernization initiatives aligned with enterprise technology and data strategies.
• Design and implement Databricks Serverless capabilities across the platform.
• Establish governance and monitoring frameworks to track Serverless consumption and ROI.
• Lead enterprise-wide FinOps adoption across the EDP ecosystem.
• Define and implement a FinOps operating model.
• Establish cost accountability across business and engineering teams.
• Design tagging standards and policy controls.
• Implement automated cost governance within DevOps deployment pipelines.
• Enable user-level and project-level cost visibility and reporting.
• Implement budget controls, spending guardrails, and proactive alerting mechanisms.
• Architect the expansion of the Enterprise Data Platform to support POSIT/RStudio capabilities.
• Design a consolidated analytics and data science platform on Databricks.
• Enable notebook-based development, data science experimentation, MLOps, API integrations, and application deployment.
• Support external data ingestion, scheduling, BI connectivity, developer tooling, and operational reporting.
• Enable AI and LLM integration patterns.
• Develop migration strategies for POSIT workloads.

Skills

Databricks architecture

Platform engineering

DevOps

FinOps

Data engineering

Stakeholder management

Governance & security compliance

Cloud cost optimization

Tools

Azure Cloud Services

Terraform

CI/CD pipelines

Job description

Role Title: Data Platform Architect - Databricks
Location: London/2-3 days a week onsite
Duration: 31/12/2026

Role Description:

• Platform Architect to lead the evolution of our Enterprise Data Platform (EDP), driving platform engineering with Strong expertise in Databricks architecture and platform administration.
• This role is ideal for an architect who can combine strategic architecture leadership with DevOps, FinOps, and Data Engineering expertise to deliver measurable business value, modernize the EDP landscape, and drive significant operational and cost efficiencies.

Key Responsibilities

Lead platform modernization initiatives aligned with enterprise technology and data strategies.

Databricks Serverless Enablement

• Lead the design and implementation of Databricks Serverless capabilities across the platform.
• Establish governance and monitoring frameworks to track Serverless consumption and ROI.
• Establish workload governance

FinOps Enablement & Cost Governance

• Lead enterprise-wide FinOps adoption across the EDP ecosystem.
• Define and implement a FinOps operating model.
• Establish cost accountability across business and engineering teams.
• Design tagging standards and policy controls.
• Implement automated cost governance within DevOps deployment pipelines.
• Enable user-level and project-level cost visibility and reporting.
• Implement budget controls, spending guardrails, and proactive alerting mechanisms.

Discovery Zone Enhancement & POSIT Migration

• Architect the expansion of the Enterprise Data Platform to support capabilities currently delivered through POSIT/RStudio.
• Design a consolidated analytics and data science platform on Databricks.
• Enable notebook-based development, data science experimentation, MLOps, API integrations, and application deployment.
• Support external data ingestion, scheduling, BI connectivity, developer tooling, and operational reporting.
• Enable AI and LLM integration patterns.
• Develop migration strategies for workloads currently hosted on POSIT.

DevOps & Platform Engineering

• Define and implement platform engineering standards and self-service capabilities.
• Design Infrastructure as Code (IaC) solutions using Terraform and cloud-native tooling.
• Implement CI/CD pipelines for platform deployments and data engineering workloads.
• Automate platform provisioning, governance, monitoring, and compliance controls.
• Integrate security and policy enforcement into DevSecOps processes.
• Establish platform observability using monitoring and logging frameworks.

Essential

• SFIA level 6 and above
• Hands-on experience with Azure cloud services.
• Deep understanding of DevOps, CI/CD, Infrastructure as Code, and automation frameworks.
• Proven experience architecting large-scale cloud-based data platforms.
• Strong expertise in Databricks architecture and platform administration.
• Experience implementing FinOps practices and cloud cost optimization strategies.
• Strong understanding of data governance, security, and compliance requirements.
• Experience leading platform transformation and modernization programmes.
• Stakeholder management experience across technical and business functions.

Technical Skills

• Azure and Terraform
• Platform Engineering
• Databricks Serverless capabilities.
• Cloud administration
• Platform cost management.
• Governance and compliance controls.
• Manage platform operational overhead.
• Manage platform reliability and performance.
• Manage security and supportability.