Senior Tech Lead, Azure Databricks Data Platform

Eaglecliff Recruitment; Greater London; On-site; GBP 110,000 - 150,000; Full time

Job summary

Eaglecliff Recruitment is seeking a Tech Lead to drive data engineering initiatives for a Global Energy Company on Azure Databricks. The role focuses on delivering scalable streaming data solutions using Structured Streaming, Delta Live Tables, PySpark, Python and SQL.

You will lead design and delivery, optimise runtime and cloud costs, and mentor teams with 10+ years in data engineering, including 3–5 years in leadership within energy trading contexts.

Qualifications

• 10+ years in data engineering, with 3–5 years leading teams.
• Experience in the financial trading domain with trading data models.
• Azure Databricks, DLT, PySpark, Python and SQL experience.
• Experience in Energy/Commodities Trading environments.
• Degree in Computer Science/Engineering or equivalent.
• Azure or Databricks certifications.

Responsibilities

• Leading the design, development, and implementation of scalable, high-performance data solutions on Azure and Databricks.
• Optimising platform runtime performance and managing cloud costs, with a focus on Databricks streaming workloads.
• Providing end-to-end technical leadership, from solution architecture through delivery and production support.

Skills

Azure Databricks

Delta Live Tables

PySpark

Python

SQL

Structured Streaming

Trading data models

Energy trading environments

Leadership

Education

Degree in Computer Science/Engineering or equivalent

Job description

Eaglecliff Recruitment is seeking a Tech Lead to drive data engineering initiatives for a Global Energy Company on Azure Databricks. The role focuses on delivering scalable streaming data solutions using Structured Streaming, Delta Live Tables, PySpark, Python and SQL.

You will lead design and delivery, optimise runtime and cloud costs, and mentor teams with 10+ years in data engineering, including 3–5 years in leadership within energy trading contexts.