Senior Data Engineer | Energy Trading | Agnostic - Cloud-native AI Centric Data Platform | Next[...]

VirtueTech Recruitment Group; England; On-site; GBP 90,000 - 110,000; Full time

Job summary

A renowned global energy trading company is seeking a Senior Data Engineer to lead the development of a cloud-native, AI-driven analytical data platform. You will design and build high-performance ClickHouse and Azure Databricks data clusters for trading and analytics teams. The role offers a competitive salary of up to £100K plus bonus and benefits. If you have experience in data architecture and ingestion frameworks, this is the perfect opportunity to make an impact in a dynamic environment.

Qualifications

• Experience in designing high-performance data clusters.
• Strong background in cloud-native architecture.
• Proficiency in building scalable data solutions.

Responsibilities

• Architect and maintain a modern lakehouse platform.
• Develop ingestion frameworks for various data types.
• Build scalable ETL/ELT pipelines for trading teams.

Skills

ClickHouse

Azure Databricks

Cloud-native data platforms

Python

Data architecture

ETL/ELT pipelines

Job description

Senior Data Engineer | Energy Trading | Agnostic - Cloud-native AI Centric Data Platform | Next-Gen Lakehouse | Up to £100K + Bonus + Benefits

Senior Data Engineer is required for a leading global Energy & Commodities trading house — one of the world’s largest and most diverse general trading companies. Their trading ecosystem spans multiple commodities and markets, creating high-volume, high-velocity data challenges ideally suited to ClickHouse and modern data engineering.

As a Senior Data Engineer, you will play a pivotal role in choosing the right tooling, building and scaling a cloud-native AI-driven analytical data platform. You’ll be a senior member of the Core Engineering team, shaping the architecture and implementing high-performance data solutions used directly by trading, risk, analytics and operational teams.

Whilst the firm has agreed to do an Azure Databricks POC platform in Q1 2026, with so much of their data on‑prem, they are still showcasing different Cloud-native platforms, and therefore are looking for someone who is excited to build Proof of Concepts in the right platform for the business requirements.

This is a hands‑on position where you’ll be working with a high-skilled small team of engineers while designing and developing a next‑generation lakehouse architecture.

Azure Databricks & ClickHouse will form the backbone of examples of two systems, supporting ultra-fast analytics, market-data ingestion, tick‑data storage, and real‑time insights for traders and quantitative teams.

• Salary: Up to £100,000 per annum + Bonus & Benefits

What You’ll Do

• Design, build, and optimise high-performance ClickHouse / Azure Databricks / AI Cloud-native Data clusters for large‑scale analytical workloads across commodities and energy trading.
• Develop ingestion frameworks for market data, pricing data, tick data, and trade/position datasets, leveraging ClickHouse’s columnar storage and real‑time querying.
• Architect and maintain a modern lakehouse platform integrating ClickHouse with Databricks, Snowflake, and Azure-based services.
• Define and enforce best practices for data modelling, partitioning, compression, and query optimisation tailored for ClickHouse.
• Build scalable Python-based ETL/ELT pipelines and data services that deliver robust, low-latency data to trading, risk, and analytics teams.

If you are interested in this role, please send your CV to [e-mail].

Location: London, England, United Kingdom

Referrals increase your chances of interviewing at VirtueTech Recruitment Group by 2x.