Azure Databricks Data Engineer

Marks Sattin (UK) Ltd; City Of London; Hybrid; GBP 120,000 - 166,000; Full time

Job summary

Marks Sattin (UK) Ltd partners with an international digital infrastructure organisation to hire an Azure Databricks Data Engineer on a 6-month contract in London, with hybrid onsite/remote working. The role focuses on designing and delivering Databricks-based data solutions to support reporting, analytics, and future AI initiatives.

You will collaborate with architects, engineers, and senior stakeholders to build scalable data pipelines, maintain Delta Lake and Unity Catalog implementations,

Qualifications

• Proven experience designing and delivering solutions within Azure Databricks environments.
• Strong Python, PySpark, SQL, and T-SQL development skills.
• Expertise building and supporting large-scale ETL/ELT pipelines on cloud platforms.
• Understanding Delta Lake, Unity Catalog and data governance principles.

Responsibilities

• Design, develop, and optimise Azure Databricks solutions using PySpark, Python, SQL, and Azure-native technologies.
• Build and maintain enterprise-grade ETL/ELT pipelines supporting ingestion, transformation, CDC, validation, and data delivery across multiple business functions.
• Develop and support Lakeflow pipelines, Delta Lake implementations, and modern data engineering frameworks.
• Drive best practices across performance optimisation, security, governance, monitoring, and platform reliability.
• Collaborate with business, data, and technology teams to deliver high-quality data products that support organisational growth and strategic decision-making.

Skills

Azure Databricks

PySpark

Python

SQL

T-SQL

ETL/ELT pipelines

Delta Lake

Unity Catalog

REST APIs

Git

CI/CD

Tools

Azure Data Factory

ADLS

Key Vault

Git

CI/CD

Job description

Azure Databricks Data Engineer (Contract) | International Digital Infrastructure Organisation | London (Hybrid)

6-Month Initial Contract | Inside IR35 | 4 Days Onsite, 1 Day Remote

We're partnering with a leading international digital infrastructure organisation undergoing a significant data transformation programme. Operating across multiple markets and supporting critical communications infrastructure, the business is investing heavily in its cloud, data, and analytics capabilities to enable more informed decision-making and unlock greater value from its enterprise data estate.

This is an opportunity to join a high-performing Digital Innovations team at a pivotal stage of that journey. You'll play a key role in building and enhancing a modern Azure Databricks platform that supports business-critical reporting, analytics, and future AI initiatives. Working closely with architects, engineers, and senior stakeholders, you'll help shape the direction of the data platform while delivering scalable, secure, and high-quality data solutions.

What You'll Do

• Design, develop, and optimise Azure Databricks solutions using PySpark, Python, SQL, and Azure-native technologies.
• Build and maintain enterprise-grade ETL/ELT pipelines supporting ingestion, transformation, CDC, validation, and data delivery across multiple business functions.
• Develop and support Lakeflow pipelines, Delta Lake implementations, and modern data engineering frameworks.
• Drive best practices across performance optimisation, security, governance, monitoring, and platform reliability.
• Collaborate with business, data, and technology teams to deliver high-quality data products that support organisational growth and strategic decision-making.

What You'll Bring

• Proven experience designing and delivering solutions within Azure Databricks environments.
• Strong Python, PySpark, SQL, and T-SQL development skills.
• Expertise building and supporting large-scale ETL/ELT pipelines and cloud-based data platforms.
• Strong understanding of Delta Lake, Unity Catalog, data governance principles, and medallion architecture.
• Experience integrating REST APIs, JSON data sources, Azure Data Factory, ADLS, Key Vault, and other Azure services.
• Hands-on experience with Git, CI/CD pipelines, deployment automation, and production support.
• Strong troubleshooting, performance tuning, and optimisation experience across Databricks and SQL environments.
• Excellent stakeholder engagement and communication skills, with the ability to operate effectively across both technical and business teams.

Desirable Experience

• Exposure to Microsoft Fabric.
• Databricks certifications.
• Experience with Terraform and infrastructure-as-code practices.
• Experience working within large, international or regulated environments.

The Opportunity

This role offers the chance to join a business where data is becoming increasingly central to operational performance and strategic growth. You'll have the opportunity to influence a core enterprise platform, work on complex and meaningful data challenges, and contribute to a long-term transformation programme with real organisational impact.

We are committed to protecting the privacy of all our candidates and clients, please visit https://www.markssattin.co.uk/privacy and https://uk.grafton-recruitment.com/en/privacy-policy-1 for our privacy policy.