Contract Tech Lead : Azure Databricks Data Platform

Eaglecliff Recruitment; Greater London; On-site; GBP 110,000 - 150,000; Full time

Job summary

Eaglecliff Recruitment is seeking a Tech Lead to drive data engineering initiatives for a Global Energy Company on Azure Databricks. The role focuses on delivering scalable streaming data solutions using Structured Streaming, Delta Live Tables, PySpark, Python and SQL.

You will lead design and delivery, optimise runtime and cloud costs, and mentor teams with 10+ years in data engineering, including 3–5 years in leadership within energy trading contexts.

Qualifications

• 10+ years in data engineering, with 3–5 years leading teams.
• Experience in the financial trading domain with trading data models.
• Azure Databricks, DLT, PySpark, Python and SQL experience.
• Experience in Energy/Commodities Trading environments.
• Degree in Computer Science/Engineering or equivalent.
• Azure or Databricks certifications.

Responsibilities

• Leading the design, development, and implementation of scalable, high-performance data solutions on Azure and Databricks.
• Optimising platform runtime performance and managing cloud costs, with a focus on Databricks streaming workloads.
• Providing end-to-end technical leadership, from solution architecture through delivery and production support.

Skills

Azure Databricks

Delta Live Tables

PySpark

Python

SQL

Structured Streaming

Trading data models

Energy trading environments

Leadership

Education

Degree in Computer Science/Engineering or equivalent

Job description

Exciting opportunity for a Tech Lead with experience delivering projects on Azure Databricks to join a long term project at a Global Energy Company.

The role requires strong hands-on experience with Structured Streaming, Delta Live Tables (DLT), PySpark architecture, Python and SQL.

Key Responsibilities

• Leading the design, development, and implementation of scalable, high-performance data solutions on Azure and Databricks
• Optimising platform runtime performance and managing overall cloud costs, with a strong focus on Databricks streaming workloads
• Providing end-to-end technical leadership, from solution architecture through delivery and production support

Experience & Skills

• 10+ years in data engineering, with at least 3–5 years leading teams
• Must have experience within the financial trading domain, including knowledge of trading data models and related business processes
• Experience with Azure Databricks, DLT, Pyspark, Python and SQL
• Experience within Energy / Commodities Trading Environments
• Degree in Computer Science/Engineering or equivalent
• Azure or Databricks certifications