Lead Data Engineer, Azure Lakehouse Platform

BMS Group; City Of London; On-site; GBP 76,500 - 93,500; Full time

Job summary

A major insurance firm in London is seeking a skilled Senior Data Engineer to manage and optimize data engineering pipelines. You will lead a team to implement best practices in a Lakehouse platform within Azure, liaising closely with business stakeholders. The ideal candidate has strong experience in leading data engineering efforts, managing teams, and is proficient with SQL, Python, and Azure tools.

Qualifications

• Experience working as a principal/lead Data Engineer.
• Experience working with large data sets and proficiency in SQL, Python and PySpark.
• Experience managing a team of engineers.

Responsibilities

• Own and manage all data engineering pipelines.
• Manage the prioritization of data engineering backlog.
• Continuously engage with stakeholders to understand business needs.

Skills

SQL

Python

PySpark

Data Engineering

Communication

Tools

Azure Databricks

Metadata-driven approaches

Job description

A major insurance firm in London is seeking a skilled Senior Data Engineer to manage and optimize data engineering pipelines. You will lead a team to implement best practices in a Lakehouse platform within Azure, liaising closely with business stakeholders. The ideal candidate has strong experience in leading data engineering efforts, managing teams, and is proficient with SQL, Python, and Azure tools.