Contract Technical Lead (Azure Databricks Platform)

X4 Technology; Greater London; On-site; GBP 90,000 - 140,000; Part time

Job summary

X4 Technology is seeking an experienced Contract Technical Lead to steer the Azure Databricks data platform within a major energy trading transformation. You will define technical direction, drive architecture decisions, optimise performance and enforce engineering standards across the programme.

Join a multi-year migration from Oracle to Azure Databricks, delivering real-time analytics, governance, and scalable data platforms for global Endur reporting.

Qualifications

• Proven experience leading the design and delivery of enterprise-scale Azure Data Platform and Databricks solutions.
• Strong hands-on expertise with Azure Databricks, Structured Streaming, Delta Live Tables, PySpark, Python and SQL, with a focus on performance optimisation.
• Strong understanding of cloud data architecture, data governance, data quality and engineering best practices.
• Excellent leadership, stakeholder management and communication skills, with experience mentoring teams and driving technical delivery.

Responsibilities

• Lead the architecture, design and delivery of scalable Azure Databricks data solutions, providing technical leadership from solution design through to production.
• Drive best practices across data engineering, platform architecture, governance, security and engineering standards, mentoring teams and influencing technical direction.
• Optimise Databricks performance, including Structured Streaming, Delta Live Tables and cloud cost efficiency.
• Partner with business stakeholders, architects and engineering teams to translate complex requirements into scalable, high-performance data solutions.
• Lead the migration of legacy enterprise data platforms into Azure Databricks, delivering a unified data platform for Endur, global reporting and analytics while shaping its long-term evolution.

Skills

Azure Databricks

Structured Streaming

Delta Live Tables

PySpark

Python

SQL

Cloud data architecture

Leadership

Stakeholder management

Job description

Contract:

Initial 6-month contract (long-term transformation programme)

X4 Technology are partnered with a leading global energy trading organisation delivering a major transformation, centred on the migration from a legacy ETRM platform to Endur and the modernisation of their enterprise data landscape through Azure Databricks.

As part of this multi-year programme, the business is migrating Oracle-based data platforms into Azure Databricks to create a scalable data platform supporting Endur, global reporting, analytics and risk, while reducing critical data latency from 10-15 minutes to near real-time.

They're looking for an experienced Technical Lead to define the technical direction of the Azure Databricks platform, driving architecture, performance optimisation and engineering best practices across one of the organisation's most strategic cloud modernisation programmes.

Responsibilities for the Contract Technical Lead - Azure Databricks Data Platform

• Lead the architecture, design and delivery of scalable Azure Databricks data solutions, providing technical leadership from solution design through to production.


• Drive best practices across data engineering, platform architecture, governance, security and engineering standards, mentoring teams and influencing technical direction.


• Optimise Databricks performance, including Structured Streaming, Delta Live Tables and cloud cost efficiency.


• Partner with business stakeholders, architects and engineering teams to translate complex requirements into scalable, high-performance data solutions.


• Lead the migration of legacy enterprise data platforms into Azure Databricks, delivering a unified data platform for Endur, global reporting and analytics while shaping its long-term evolution.

Requirements for the Contract Technical Lead - Azure Databricks Data Platform

• Proven experience leading the design and delivery of enterprise-scale Azure Data Platform and Databricks solutions.


• Strong hands-on expertise with Azure Databricks, Structured Streaming, Delta Live Tables, PySpark, Python and SQL, with a focus on performance optimisation.


• Strong understanding of cloud data architecture, data governance, data quality and engineering best practices.


• Excellent leadership, stakeholder management and communication skills, with experience mentoring teams and driving technical delivery.


• Experience within energy trading, commodities, capital markets or wider financial services, with knowledge of trading data models and business processes highly desirable.

This is an excellent opportunity share the technical direction of a business-critical Azure Databricks platform for one of the world's biggest names within energy trading.