Cloud Data Engineer – Azure Databricks & Pipelines (Hybrid)

Lorien

Greater London

Hybrid

GBP 92,000 - 138,000

Part time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Lorien is seeking an experienced Data Engineer for a 6‑month contract in London. The role is hybrid with 50% onsite work and requires Active SC Clearance. The project focuses on designing and delivering a modern, cloud‑first data platform for a leading UK financial sector organisation.

You will design scalable data solutions on Azure, develop robust pipelines with Python/SQL/Spark, and lead data governance and quality initiatives while modernising legacy systems and guiding migrations to Azure.

Qualifications

  • Extensive experience with Azure data services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory.
  • Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing.
  • Proven experience in CI/CD practices using industry‑standard tools (eg, GitHub Actions, Azure DevOps).
  • Strong understanding of data architecture principles and cloud‑native design patterns.

Responsibilities

  • Design, build and deploy scalable, secure data solutions using Azure Databricks, Data Factory and Data Lake Storage.
  • Develop and optimise advanced data pipelines with Python, SQL, Spark/PySpark and Delta Lake.
  • Champion strong data quality, governance and observability practices.
  • Modernise Legacy systems and support large‑scale migrations into Azure.
  • Provide technical leadership, set engineering standards and contribute to architectural decisions.
  • Mentor engineers and foster a culture of continuous improvement.
  • Work closely with architecture, analytics and business teams to align engineering solutions with organisational needs.

Skills

SQL
Python
Spark/PySpark
Linux/Unix
Shell scripting

Tools

Azure Databricks
Azure Data Lake Storage
Azure Data Factory
GitHub Actions
Azure DevOps
Delta Lake
Apache Airflow

Job description

Lorien is seeking an experienced Data Engineer for a 6‑month contract in London. The role is hybrid with 50% onsite work and requires Active SC Clearance. The project focuses on designing and delivering a modern, cloud‑first data platform for a leading UK financial sector organisation.

You will design scalable data solutions on Azure, develop robust pipelines with Python/SQL/Spark, and lead data governance and quality initiatives while modernising legacy systems and guiding migrations to Azure.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Azure Data Engineer: Build Data Pipelines (Hybrid, London)
Azure Data Engineer: Build Data Pipelines (Hybrid, London)

Morson Group • Greater London

Hybrid
GBP 55,000 - 63,000
Data Engineer - SC Cleared
Data Engineer - SC Cleared

Lorien • Greater London

Hybrid
GBP 92,000 - 138,000
Hybrid Data Engineer – Azure Data Pipelines (London)
Hybrid Data Engineer – Azure Data Pipelines (London)

Pearson Carter • Greater London

Hybrid
GBP 63,000 - 77,000
Hybrid working model
London office presence
Enterprise-scale data projects
+2
London Hybrid Senior Azure Data Engineer - Databricks
London Hybrid Senior Azure Data Engineer - Databricks

Capco • Greater London

Hybrid
GBP 90,000 - 120,000
Discretionary bonus
Health insurance
Life insurance
+2
Data Engineer: Azure Databricks & Cloud Pipelines (Hybrid)
Data Engineer: Azure Databricks & Cloud Pipelines (Hybrid)

Standard Life plc • City of Edinburgh

Hybrid
GBP 35,000 - 45,000
Bonus
Private medical cover
38 days annual leave
+5
Azure Data Engineer - Hybrid London | Scalable Pipelines
Azure Data Engineer - Hybrid London | Scalable Pipelines

Recann • London

Hybrid
GBP 63,000 - 77,000
Competitive salary
Inclusive culture
Data Engineer - Databricks & Azure (London, Contract)
Data Engineer - Databricks & Azure (London, Contract)

Hydrogen Group • Greater London

Hybrid
GBP 40,000 - 60,000
Azure Databricks Data Engineer — Hybrid London
Azure Databricks Data Engineer — Hybrid London

Osmii • London

Hybrid
GBP 70,000 - 90,000
Databricks Tech Lead - Azure Data Platform Delivery (Hybrid)
Databricks Tech Lead - Azure Data Platform Delivery (Hybrid)

Data & AI • Greater London

Hybrid
GBP 96,000 - 126,000
Cloud Data Engineer - Azure, Python & SQL
Cloud Data Engineer - Azure, Python & SQL

VM2R Services Ltd • West of England

On-site
GBP 50,000 - 75,000
Competitive salary
Discretionary bonus
25 days holiday pro rata
+2