Senior Data Engineering Lead (Databricks & Azure)

KPMG Careers

Birmingham

On-site

GBP 90,000 - 140,000

Full time

43 hours ago

Be an early applicant

Application generator

A complete application in a minute — tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

KPMG in the UK seeks a Senior Manager - Principal Data Engineer to shape data engineering strategy across Technology & Solutions. You will lead enterprise-scale Databricks-based solutions on Azure, drive modern engineering practices, and mentor teams to deliver data-enabled insights across business units.

Key duties include designing scalable data models, governing Delta Lake architectures, and integrating BI tools like Power BI.

Qualifications

Advanced, hands-on Databricks platform expertise including PySpark, Delta Lake storage, and Unity Catalog governance.
Extensive experience in designing advanced data models including medallion architectures and enterprise data warehouses.
Expert-level SQL for complex data transformations and analytics workloads.
Advanced Python for data engineering, pipeline automation, and integrations.
Proven cloud platform experience with strong preference for Microsoft Azure.
Experience with ETL/ELT development and data pipeline orchestration (Databricks Workflows, DLT, ADF).
Extensive experience with Git version control, GitHub collaboration, and modern CI/CD for data workloads.
Agile software development (Scrum/Kanban) and DevOps practices applied to data engineering contexts.
Balance technical execution with people management and product accountability.
Experience implementing data governance, data lineage, RBAC (Atlas/Purview).
Event-driven real-time data pipelines using Kafka/AWS Kinesis/Azure Event Hubs.
Familiarity with AI frameworks and agentic AI architectures.

Responsibilities

Design and deliver enterprise-scale data solutions on cloud platforms (Azure preferred), leveraging Databricks as the core data engineering platform.
Build and optimize ETL/ELT data pipelines using PySpark, SQL, Python, orchestrating workflows with Databricks Workflows, Delta Live Tables, Azure Data Factory.
Implement and maintain Delta Lake storage architectures and Unity Catalog governance across the data estate.
Design scalable, performant, reliable data solutions with enterprise-grade standards from inception to production.
Lead technical discovery and translate business needs into data platform roadmaps.
Integrate data platforms with BI tools including Databricks AI/BI and Power BI for self-service analytics.
Establish data engineering standards, coding practices, and quality gates for maintainable platforms.
Champion modern software engineering practices including Git, CI/CD, automated testing for data workloads.
Drive observability, monitoring, and alerting for data pipelines and platforms.
Promote AI-augmented data engineering while maintaining code quality.