Senior Data Engineer - Databricks & Azure Lead

Howden

Greater London

Hybrid

GBP 70,000 - 115,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Hybrid working
Collaborative culture

Job summary

Howden is seeking a Senior Data Engineer to own high-impact data initiatives in a modern Azure-based data ecosystem. You will design scalable metadata-driven pipelines using Databricks and Azure services, while guiding junior engineers and collaborating with data scientists and analysts to deliver business insights.

The role emphasizes ownership of complex technical problems, robust data solutions, and adherence to CI/CD and security best practices within a hybrid work environment.

Qualifications

  • 4-6 years of data engineering with Databricks focus (SQL & PySpark).
  • Strong hands-on with Azure data services (ADF, ADLS, Functions).
  • Proven experience designing metadata-driven data pipelines.
  • Understanding of orchestration and data workflow automation (Airflow, DBT, Spark Declarative Pipelines).
  • Strong CI/CD practices for data engineering.

Responsibilities

  • Take ownership of metadata ingestion frameworks and orchestration processes.
  • Design and implement scalable metadata-driven services for data pipelines.
  • Build and maintain data solutions using Databricks and Azure services.
  • Embed best practices for CI/CD and DevOps for data.
  • Collaborate with stakeholders to translate data needs into engineering solutions.
  • Optimize data pipelines for performance, reliability, and scalability.
  • Support data governance and security best practices.
  • Design, build, and maintain ETL processes and analytical data products.
  • Provide mentoring to junior engineers and review code.

Skills

Databricks (SQL & PySpark)
Azure data services
Airflow
DBT
CI/CD for data engineering

Tools

Azure Data Factory
Azure Data Lake Storage
Azure Functions

Job description

Howden is seeking a Senior Data Engineer to own high-impact data initiatives in a modern Azure-based data ecosystem. You will design scalable metadata-driven pipelines using Databricks and Azure services, while guiding junior engineers and collaborating with data scientists and analysts to deliver business insights.

The role emphasizes ownership of complex technical problems, robust data solutions, and adherence to CI/CD and security best practices within a hybrid work environment.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer — Databricks on Azure (Hybrid/Flexible)
Senior Data Engineer — Databricks on Azure (Hybrid/Flexible)

Howden Group • England

Hybrid
GBP 75,000 - 110,000
Senior Data Engineer | Databricks & Azure | Hybrid
Senior Data Engineer | Databricks & Azure | Hybrid

TigerRisk Partners, LLC. • Greater London

Hybrid
GBP 70,000 - 110,000
Hybrid working
Senior Data Engineer: Databricks, Azure & Metadata-Driven
Senior Data Engineer: Databricks, Azure & Metadata-Driven

Howden Group Holdings • City Of London

Hybrid
GBP 90,000 - 120,000
Senior Data Engineer: Azure/Databricks Tech Lead
Senior Data Engineer: Azure/Databricks Tech Lead

Hyperiongrp • Greater London

Hybrid
GBP 85,000 - 120,000
Hybrid working
Reasonable accommodations
Senior Data Engineer - Databricks & Azure, Hybrid Role
Senior Data Engineer - Databricks & Azure, Hybrid Role

Howden Group Services Limited • City Of London

On-site
GBP 80,000 - 120,000
Lead Data Engineer: Azure Databricks & Metadata Pipelines
Lead Data Engineer: Azure Databricks & Metadata Pipelines

Howden Group Holdings • England

Hybrid
GBP 90,000 - 140,000
Senior Data Engineer - Databricks & Azure Lead (Hybrid)
Senior Data Engineer - Databricks & Azure Lead (Hybrid)

vavemga • Manchester

Hybrid
GBP 70,000 - 110,000
Hybrid working
Competitive base salary
Pension
+1
Senior Azure Data & AI Platform Engineer
Senior Azure Data & AI Platform Engineer

Howden Group Holdings • City Of London

Remote
GBP 90,000 - 130,000
Remote-first
Learning budget
Equipment provided
+1
Senior Azure Data & AI Platform Engineer - Remote
Senior Azure Data & AI Platform Engineer - Remote

Howden Group Holdings • Greater London

Hybrid
GBP 110,000 - 150,000
Remote-first
Learning budget
Equipment & licenses
+1
Azure Tech Lead — Senior Engineer
Azure Tech Lead — Senior Engineer

London Insurance Life • Greater London

Hybrid
GBP 80,000 - 90,000