Senior Azure Databricks Data Engineer - Public Sector

GIS Recruiter Ltd

Greater London

On-site

GBP 90,000 - 120,000

Full time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

GIS Recruiter Ltd seeks an experienced Senior Databricks Data Engineer to design and implement enterprise-scale data platforms in Azure Databricks for UK public sector environments. You will work with large, sensitive datasets and secure cloud architectures, delivering Delta Lake and lakehouse solutions with robust governance.

The role emphasizes PySpark, SQL optimisation, CI/CD pipelines, and collaboration with architects, analysts and government stakeholders to meet strict security and

Qualifications

  • 5+ years hands-on Databricks experience.
  • Expert Azure Databricks knowledge.
  • Strong PySpark, Spark SQL and Python.
  • Experience with Delta Lake and related Lakehouse architectures.
  • Experience implementing Medallion Architecture (Bronze, Silver, Gold).
  • Knowledge of Spark architecture, optimisation and performance tuning.
  • Unity Catalog and data governance experience.
  • Incremental data loads, CDC and SCD Type 1/2 experience.
  • SQL development and query optimisation skills.
  • Experience integrating Azure Data Factory with Databricks.

Responsibilities

  • Design scalable Databricks data pipelines.
  • Build and maintain enterprise-grade Lakehouse platforms.
  • Optimise Spark workloads and Databricks clusters.
  • Implement data quality and governance controls.
  • Collaborate with architects, analysts and stakeholders.
  • Support production deployments and troubleshoot complex data issues.
  • Ensure solutions meet public sector security and compliance requirements

Skills

Databricks
Azure Databricks
PySpark
Python
ETL Pipelines
Delta Lake
Lakehouse
Spark Tuning
Data Governance
SQL
CI/CD
Git
Azure DevOps

Tools

Azure Data Factory
Git
Azure DevOps

Job description

GIS Recruiter Ltd seeks an experienced Senior Databricks Data Engineer to design and implement enterprise-scale data platforms in Azure Databricks for UK public sector environments. You will work with large, sensitive datasets and secure cloud architectures, delivering Delta Lake and lakehouse solutions with robust governance.

The role emphasizes PySpark, SQL optimisation, CI/CD pipelines, and collaboration with architects, analysts and government stakeholders to meet strict security and

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Databricks Data Engineer for Public Sector Platform
Senior Databricks Data Engineer for Public Sector Platform

GIS Recruiter Ltd • City Of London

On-site
GBP 90,000 - 130,000
Databricks Data Engineer
Databricks Data Engineer

GIS Recruiter Ltd • Greater London

On-site
GBP 90,000 - 120,000
Databricks Data Engineer
Databricks Data Engineer

GIS Recruiter Ltd • City Of London

On-site
GBP 90,000 - 130,000
Azure Databricks Lakehouse Architect: Governance & Security
Azure Databricks Lakehouse Architect: Governance & Security

GIS Recruiter Ltd • City Of London

On-site
GBP 120,000 - 165,000
Databricks Solution Architect: Lakehouse & Governance Lead
Databricks Solution Architect: Lakehouse & Governance Lead

GIS Recruiter Ltd • Greater London

On-site
GBP 90,000 - 130,000
Databricks Solution Architect
Databricks Solution Architect

GIS Recruiter Ltd • City Of London

On-site
GBP 120,000 - 165,000
Databricks Solution Architect
Databricks Solution Architect

GIS Recruiter Ltd • Greater London

On-site
GBP 90,000 - 130,000
Senior Data Engineer
Senior Data Engineer

Ceox • United Kingdom

Remote
GBP 83,000 - 131,000
Senior Data Engineer (Remote)
Senior Data Engineer (Remote)

Ceox Services Ltd • United Kingdom

Remote
GBP 92,000 - 129,000
Senior Data Engineer — Azure, Databricks & Lakehouse
Senior Data Engineer — Azure, Databricks & Lakehouse

DCG (Poland) • United Kingdom

Remote
GBP 80,000 - 120,000
Private medical care
Co-financing for the sports card
Dedicated consultant support