Principal Data Engineer

Deloitte - Recruitment

Greater London

On-site

GBP 120,000 - 175,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Deloitte is seeking a Principal Data Engineer to lead the delivery of Azure-based data lake solutions in London. You will drive governance, performance, and cross-functional collaboration for high-volume data environments, guiding Databricks Delta Live Tables, Unity Catalog, and related technologies.

The role focuses on architectural leadership, data transformation pipelines, and ensuring cost-effective cloud data platforms with strong executive communication.

Qualifications

  • Bachelor’s or Master’s degree in IT or a related discipline.
  • 15–17 years of IT experience with cloud-based data analytics.
  • Proven data lake architectures experience and performance optimization.
  • Azure/Databricks certification preferred.
  • Strong Azure tech knowledge (ADF/ADLS/Databricks/Stream Analytics/Synapse).
  • SQL and Python/Scala/Spark-SQL proficiency.
  • Experience with ETL pipelines and data migration to cloud.

Responsibilities

  • Architect and implement scalable Azure-based data lake solutions.
  • Lead capability-building roadmaps across Qlik Replicate, Databricks Delta Live Tables, and Unity Catalog.
  • Provide technical leadership and ensure best practices/quality.
  • Develop data transformation pipelines with Delta Live Tables and Structured Streaming.
  • Optimize Databricks resources for high-volume processing within budget.
  • Govern data with Unity Catalog, governance, lineage, and metadata management.
  • Enforce robust data governance, security, and data quality.
  • Monitor pipelines and resolve production issues; align projects with business goals.
  • Mentor teams in Azure, Databricks, and data streaming technologies.
  • Lead knowledge transfer sessions on cutting-edge methodologies.

Education

Bachelor’s or Master’s degree in IT or a related discipline

Tools

Azure Data Factory (ADF)
Azure Data Lake Storage (ADLS)
Databricks
Delta Live Tables
Unity Catalog
Synapse Analytics
Qlik Replicate
PowerBI / QlikSense
Hadoop
Kafka
Spark

Job description

Job Description

Role: Principal Data Engineer

Location: London Onsite Full Time

Start Date: ASAP

End Date: 12 months

Daily Rate Inside IR35

Payroll provider – Rockford Payroll Info for Contingent Workers – Rockford Pay

Assignment Overview

Deloitte sets the requirements for the assignment. Rockford Resourcing acts as the agency on behalf of Deloitte and is responsible for sourcing candidates to fulfil the assignment. Successful candidates will be employed by Rockford Payroll. Rockford Payroll is responsible for the Associate’s employment, pay and applicable employment-related processes.

The Role

We are seeking a visionary Principal Data Engineer to lead the delivery of innovative data analytics solutions, with a focus on technical excellence, governance, and operational performance in a high-volume data environment. This role involves strategic and technical leadership in building Azure-based data lake solutions, driving maturity in data management, and engaging with cross-functional teams to leverage data for organizational success.

Key Responsibilities
  • Architect, design, and implement scalable Azure-based data lake solutions.
  • Establish roadmaps for capability-building across technologies such as Qlik Replicate, Databricks Delta Live Tables, and Unity Catalog.
  • Provide technical leadership, ensuring adherence to best practices and quality standards.
  • Lead the development of data transformation pipelines using Databricks Delta Live Tables and Structured Streaming.
  • Optimize Databricks resources for high-volume data processing without exceeding budget.
  • Oversee the use of Databricks Unity Catalog for data governance, lineage, and metadata management.
  • Develop and enforce robust data governance policies, ensuring compliance, security, and data quality.
  • Monitor data pipeline performance and proactively resolve production issues.Collaborate with cross-functional teams to align project objectives with business goals.
  • Translate technical challenges into actionable insights for executive teams.
  • Mentor and develop a high-performing team skilled in Azure, Databricks, and data streaming technologies.
  • Facilitate knowledge transfer sessions to empower internal teams with cutting‑edge methodologies.
Experience & Skills Required
  • Bachelor’s or Master’s degree in IT or a related discipline.
  • 15–17 years of experience in the IT industry with expertise in delivering cost-effective cloud-based data analytics solutions.
  • Proven track record in building data lake architectures and optimizing performance in big data environments.
  • Certification in Azure Data Engineering or Databricks preferred.
  • Strong knowledge of Azure technologies, including ADF, ADLS, Databricks, Stream Analytics, Delta Live Tables, and Synapse Analytics.
  • Advanced proficiency in SQL and programming languages such as Python, Scala, or Spark‑SQL.
  • Experience in designing and developing scalable ETL pipelines and optimizing data workflows.
  • Deep understanding of data migration from on‑premises to cloud and cloud to cloud.
  • Proven ability to lead diverse teams and manage end‑to‑end analytics projects.
  • Exceptional communication and collaboration skills for engaging executives and stakeholders.
  • Strong sense of ownership, accountability, adaptability, and commitment to results.
  • Expertise in identifying risks and implementing effective mitigation strategies.
Nice to Have
  • Familiarity with big data tools like Hadoop, Kafka, and Spark.
  • Experience with visualization tools such as PowerBI or QlikSense.
  • Knowledge of AI/ML, NLP, and GenAI concepts.
  • Familiarity with AGILE and KANBAN methodologies.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Principal Data Engineer
Principal Data Engineer

Deloitte - Recruitment • London Borough of Lambeth

On-site
GBP 110,000 - 150,000
Principal Data Engineer
Principal Data Engineer

Nasstar • Greater London

Remote
GBP 100,000 - 150,000
Remote-first
Private health insurance
AWS & Databricks certifications
+1
Principal Data Engineer
Principal Data Engineer

Nasstar • United Kingdom

Remote
GBP 90,000 - 130,000
Remote-first working with client focus
Private health insurance
Global client projects
+2
Principal Data Engineer
Principal Data Engineer

Harnham • Greater London

Hybrid
GBP 90,000 - 110,000
Salary £90,000–£110,000 per year
Principal Data Engineer
Principal Data Engineer

WRK Digital • Leeds

On-site
GBP 70,000 - 90,000
Data Engineer
Data Engineer

Matchtech • West of England

On-site
GBP 60,000 - 90,000
Azure training
Professional development
Modern cloud platforms
+1
Data Engineer
Data Engineer

Whitehall Resources • Cambridgeshire and Peterborough

Hybrid
GBP 70,000 - 90,000
Data Engineer
Data Engineer

Talent • Greater London

Hybrid
GBP 90,000 - 120,000
Principal Data Engineer
Principal Data Engineer

Royal London Group • Alderley Edge

Hybrid
GBP 60,000 - 90,000
28 days annual leave plus bank holidays
Up to 14% employer pension contribution
Private medical insurance
+1
Data Engineering Consultant
Data Engineering Consultant

Avanade Inc. • Greater London

On-site
GBP 50,000 - 70,000