Real-Time Energy Data Engineer (Open-Source)

Norton Blake; Greater London; Hybrid; GBP 60,000 - 80,000; Full time

Job summary

A leading UK energy firm is seeking a Data Engineer to build high-performance data systems using open-source technologies. Join a data and trading team shaping the energy future, working on data ingestion pipelines, ETL processes, and real-time analytics. The ideal candidate will have strong skills in SQL and Python, experience with Apache Spark, Kafka, Docker, and a problem-solving mindset. This hybrid role offers flexibility with twice-a-week in-office requirements in London.

Qualifications

• Strong SQL and Python skills, experience with data processing libraries.
• Hands-on experience with Apache Spark and distributed computing.
• Experience with Docker, Kubernetes, and CI/CD pipelines.

Responsibilities

• Designing and building robust data ingestion pipelines.
• Creating and optimising ETL / ELT processes.
• Monitoring, troubleshooting and improving pipeline performance.

Skills

SQL

Python (pandas, NumPy, Spark)

Apache Spark

Kafka

Airflow

Docker

Kubernetes

CI/CD pipelines

Job description

A leading UK energy firm is seeking a Data Engineer to build high-performance data systems using open-source technologies. Join a data and trading team shaping the energy future, working on data ingestion pipelines, ETL processes, and real-time analytics. The ideal candidate will have strong skills in SQL and Python, experience with Apache Spark, Kafka, Docker, and a problem-solving mindset. This hybrid role offers flexibility with twice-a-week in-office requirements in London.