Databricks Platform Engineer — Hybrid (Barcelona)

Zurich Insurance Group; Madrid; Hybrid; EUR 90,000 - 120,000; Full time

Benefits offered by this job

Flexible working model

Home office allowance

Health insurance

Training programs

Stock options

Job summary

Zurich Insurance Group is seeking a Databricks Platform Engineer for our Technology Delivery Center. The role focuses on owning and evolving a scalable analytics platform powering the Global Finance Data Hub, with responsibilities spanning data ingestion, Delta Lake, and governance.

You will collaborate with data engineering, architecture, and product teams to implement CI/CD, IaC, and security controls in a regulated financial services environment.

Qualifications

• 5-8 years of experience in Data Engineering, Data Platform Engineering, or Cloud Data Architecture.
• Minimum 3 years of hands-on experience owning and managing Databricks platforms in enterprise environments.
• Strong expertise in Databricks services including Unity Catalog, Delta Lake, Delta Live Tables, Workflows, and Asset Bundles.
• Experience designing and supporting batch, streaming, API, and CDC data ingestion frameworks.
• Advanced proficiency in PySpark, Python, and Databricks SQL.
• Solid knowledge of DevOps practices including Git-based development, CI/CD pipelines, automated testing, and Infrastructure as Code (CloudFormation or CDK).
• Strong AWS experience covering S3, IAM, networking, security controls, and secrets management.
• Experience with large-scale data platforms in regulated industries; financial services experience desirable.
• Databricks and/or AWS certifications preferred.
• Experience with Airflow, Kafka/MSK, and enterprise data governance frameworks.

Responsibilities

• Own and evolve the Databricks platform underpinning the Zurich Global Finance Data Hub.
• Design, build, and maintain data ingestion frameworks across batch, streaming, API, and CDC data integration patterns.
• Develop and manage Delta Live Tables pipelines with data quality controls and schema enforcement.
• Administer and optimise Delta Lake environments, including performance tuning, schema evolution, time travel, VACUUM, and OPTIMIZE.
• Implement and manage Unity Catalog governance practices (access controls, data lineage, classification, namespaces).
• Build, deploy, and support Databricks Workflows for orchestration of multi-task pipelines.
• Establish and maintain CI/CD pipelines using Git and Databricks Asset Bundles.
• Collaborate with cloud and infrastructure teams to implement IaC (AWS CloudFormation or CDK).
• Manage authentication and authorisation mechanisms (IAM roles, instance profiles, credentials rotation).
• Monitor platform health, performance, metrics, and costs using Databricks system tables, CloudWatch, Grafana.
• Define cluster policies, compute strategies, and DBU governance controls.
• Ensure platform security (PrivateLink, VPC endpoints, row/column security).
• Partner with data engineering, finance, architecture, and product teams to deliver a scalable enterprise data platform.
• Drive platform best practices, automation, and continuous improvement across the data ecosystem.

Skills

Data engineering

Cloud data architecture

DevOps

PySpark

Python

Databricks fundamentals

Education

Bachelor's/ Master's in a relevant field

Tools

Databricks Unity Catalog

Delta Lake

Delta Live Tables

Databricks Workflows

Asset Bundles

AWS (S3, IAM)

Airflow

Kafka/MSK

Job description

Zurich Insurance Group is seeking a Databricks Platform Engineer for our Technology Delivery Center. The role focuses on owning and evolving a scalable analytics platform powering the Global Finance Data Hub, with responsibilities spanning data ingestion, Delta Lake, and governance.

You will collaborate with data engineering, architecture, and product teams to implement CI/CD, IaC, and security controls in a regulated financial services environment.