Senior Databricks Platform Engineer - Hybrid (Azure)

Gattaca; East Horsley; Hybrid; GBP 80,000 - 90,000; Full time

Benefits offered by this job

Hybrid working

25 days leave

Birthday day off

Flexible hours

Learning access

Private medical

Home workspace

Volunteer days

Lifestyle benefits

Job summary

Gattaca is seeking a Senior Platform Engineer to join its technology and data engineering team in a permanent hybrid role across Surrey and Greater London. You will design, build and optimise a Databricks data platform to support analytics, ML, and AI across the business.

You will develop batch and real-time data pipelines using Databricks, Spark and Delta Lake, contribute to platform architecture, and apply IaC to ensure reliability and scalability.

Qualifications

• Significant experience building and supporting scalable, production-grade data platforms.
• Strong hands-on expertise with Databricks, Apache Spark, Delta Lake, Python and SQL.
• Good understanding of cloud-based data engineering, ideally within Microsoft Azure environments.
• Experience with Infrastructure as Code tools such as Terraform, Bicep or similar.
• Understanding of data modelling, ETL and ELT processes, data quality, security and governance.
• Ability to collaborate effectively with technical and non-technical stakeholders.
• Experience with Unity Catalog, MLflow or Databricks Asset Bundles would be advantageous.
• Knowledge of dbt, Kafka, Event Hubs or other data transformation and streaming technologies would be beneficial.

Responsibilities

• Design, build and enhance a secure, scalable and reliable Databricks platform.
• Develop and maintain batch and real-time data pipelines using Databricks, Apache Spark and Delta Lake.
• Contribute to platform architecture, engineering standards and best practices across the data estate.
• Work with Product, Analytics and Data Science teams to deliver effective data platform solutions.
• Support platform operations, monitoring, performance optimisation and reliability.
• Apply Infrastructure as Code principles to support consistent and efficient platform management.
• Share technical knowledge and contribute to continuous improvement across the engineering team.

Skills

Databricks

Apache Spark

Delta Lake

Python

SQL

Azure

Data modelling

ETL/ELT

Security & governance

Stakeholder communication

MLflow

Unity Catalog

dbt

Kafka

Education

Bachelor's degree or equivalent in a related field

Tools

Terraform

Bicep

Databricks Asset Bundles

Kafka

Event Hubs

dbt

Job description

Gattaca is seeking a Senior Platform Engineer to join its technology and data engineering team in a permanent hybrid role across Surrey and Greater London. You will design, build and optimise a Databricks data platform to support analytics, ML, and AI across the business.

You will develop batch and real-time data pipelines using Databricks, Spark and Delta Lake, contribute to platform architecture, and apply IaC to ensure reliability and scalability.