Senior Data Engineer: Azure & Databricks Platform

Clarion Events; Greater London; On-site; GBP 90,000 - 120,000; Full time

Benefits offered by this job

25 days’ holiday plus bank holidays

End of year wellbeing shutdown (last/3

Extra day of leave for celebrations

Summer Hours on Fridays

HOW Days – one paid day per quarter

Pension Scheme

Private Medical Insurance

Health Cash Plan

Wellbeing Library

Mentoring Programme

Season Ticket Loan

Cycle to Work Schemes

Free on-site gym and showers

Free eyesight tests

Free flu vaccination on site

Job summary

Clarion Events invites a Senior Data Engineer to lead the design and optimisation of our modern data platform on Azure, with a focus on Databricks, Delta Lake, and Medallion modelling. You will build scalable pipelines, drive data governance and collaborate with analysts and data scientists across global events and digital products.

You will implement CI/CD for data, ensure security, and mentor junior engineers while advancing our data strategy and operational excellence.

Qualifications

• 5+ years in data engineering across cloud-native environments.
• Expert knowledge of Databricks, Delta Lake, Unity Catalog, ADF, Synapse, Event Hub, and Azure Data Services.
• Highly proficient in Python and SQL, with production-grade data pipelines.
• Strong understanding of Lakehouse design patterns and Medallion modelling.
• Hands-on CI/CD for data using Azure DevOps, YAML, Terraform, and Databricks tooling.
• Solid data governance, security and operational best practices.

Responsibilities

• Design, build, and optimise scalable data pipelines using Databricks and related tech.
• Develop ingestion, transformation, and orchestration workflows across diverse sources.
• Implement data quality, lineage, and governance per Medallion Architecture.
• Build Lakehouse data models to support analytics and product teams.
• Develop reusable frameworks for ingestion, validation, and monitoring.
• Ensure Databricks Unity Catalog, cluster policies, permissions, and environment separation.
• Evolve platform patterns with security and performance improvements.
• Implement monitoring, logging, and observability across pipelines and clusters.
• Contribute to CI/CD, testing, and DevOps for Databricks notebooks and SQL assets.
• Support Delta Sharing and Iceberg/Delta interoperability as needed.
• Partner with analysts and data scientists for high-quality data and AI use cases.
• Assist productionising ML pipelines and feature engineering environments.
• Collaborate cross-functionally with product, marketing, ops and tech teams.

Skills

5+ years in data engineering

Databricks

Delta Lake

Unity Catalog

ADF / Synapse

Python

SQL

CI/CD for data

Data governance

Azure DevOps

Terraform

Tools

Databricks

Delta Lake

Unity Catalog

ADF

Synapse

Event Hub

Azure Data Services

Job description

Clarion Events invites a Senior Data Engineer to lead the design and optimisation of our modern data platform on Azure, with a focus on Databricks, Delta Lake, and Medallion modelling. You will build scalable pipelines, drive data governance and collaborate with analysts and data scientists across global events and digital products.

You will implement CI/CD for data, ensure security, and mentor junior engineers while advancing our data strategy and operational excellence.