Lead Data Engineer

JR United Kingdom; City Of London; Hybrid; GBP 90,000 - 110,000; Full time

Job summary

A leading company is seeking a skilled Lead Data Engineer in London to develop a brand-new data platform within the Azure ecosystem and Databricks. This role offers a unique opportunity to shape data strategy and improve data quality in a collaborative environment focused on innovation and career growth.

Benefits

Greenfield Project

Collaborative Culture

Career Growth

Cutting-Edge Tech

Qualifications

• Extensive hands-on experience with Databricks and Azure.
• Python scripting experience for data automation.
• Knowledge of CI/CD methodologies.

Responsibilities

• Design and maintain a robust Databricks Data Platform.
• Connect APIs, databases, and data streams to the platform.
• Create deployment pipelines and automate workflows.

Skills

Databricks

Microsoft Azure data tools

Dimensional modelling

Python scripting

CI/CD methodologies

Tools

Azure Data Factory

Azure Synapse

Azure SQL

MS SQL Server

SSIS

Visual Studio

SSDT projects

Job description

Social network you want to login/join with:

Lead Data Engineer, london (city of london)

col-narrow-left

Client:

KDR Talent Solutions

Location:

london (city of london), United Kingdom

Job Category:

Other

-

EU work permit required:

Yes

col-narrow-right

Job Views:

3

Posted:

16.06.2025

Expiry Date:

31.07.2025

col-wide

Job Description:

Lead Data Engineer | Location: London (Hybrid - 1-2 day a month in office) | Salary: £90,000-£110,000

The Opportunity

Are you passionate about building cutting-edge data platforms that drive business growth? Our client is seeking a skilled and motivated Lead Data Engineer to play a key role in the creation of a brand-new data platform within the Azure ecosystem and Databricks.

This is an exciting opportunity to be at the forefront of data innovation, working within a newly formed Data & Analytics team in a long standing London Market Insurer. You’ll help shape the data strategy, improve data quality, and empower the business to make data-driven decisions.

As a Lead Data Engineer, you'll work closely with both technical and business stakeholders, leveraging your expertise to design, develop, and optimize a high-performance data platform built on Databricks. This platform will be built to scale, incorporating the latest advancements in data intelligence while supporting strategic business objectives.

Key Responsibilities

? Build & Develop – Design and maintain a robust Databricks Data Platform, ensuring performance, scalability, and availability.

? Data Pipelines – Connect APIs, databases, and data streams to the platform, implementing ETL/ELT processes.

? Data Integrity – Embed quality measures, monitoring, and alerting mechanisms.

? CI/CD & Automation – Create deployment pipelines and automate workflows.

? Collaboration – Work with stakeholders across Global IT, Data, and Broking teams to translate business requirements into technical solutions.

? Futureproofing – Drive the evolution of the data platform, ensuring adaptability for new data sources, analytical models, and emerging technologies.

What You’ll Bring

Extensive hands-on experience with Databricks, and Microsoft Azure data tools (must-have: Azure Data Factory, Azure Synapse, or Azure SQL).

Dimensional modelling expertise for analytics use cases.

Python scripting experience for data automation.

Experience with CI/CD methodologies for data platforms.

Knowledge of MS SQL Server, SSIS, Visual Studio, and SSDT projects.

Previous London Market Insurance experience

Why Join?

? Greenfield Project – Work on an all-new data platform, shaping its architecture from the ground up.

? Collaborative Culture – Engage with global teams in an agile, innovative environment.

? Career Growth – Play a pivotal role in driving data excellence within a forward-thinking business.

? Cutting-Edge Tech – Work with the latest advancements in Azure, Databricks, and Data Engineering.

This is a fantastic opportunity for a Lead Data Engineer looking to make a tangible impact. If you’re ready to take on a challenging and rewarding role, apply today!