Data Engineer — AI-Driven Pipelines & Dashboards

Octopus Energy Group; Greater London; Hybrid; GBP 70,000 - 100,000; Full time

Job summary

Octopus Energy Group's OEGEN is seeking a Data Engineer to own data pipelines from source systems to decision-ready outputs, building pipelines in Databricks, modeling data, and delivering dashboards. You will automate processes with Python, integrate systems via REST APIs, and contribute to internal tools using Streamlit, FastAPI, or HTMX.

The role emphasizes data quality, collaboration with non-technical teams, and a hybrid work model in London, with a focus on scalable, auditable data

Qualifications

• Strong SQL and Python skills for pipeline development, data modelling, and automation.
• Experience with modern data stack tooling: cloud data platform (Databricks preferred) and version control (Git/GitHub).
• Solid understanding of data modelling, warehousing patterns, and ETL/ELT design.
• Comfortable with Postgres or similar relational databases.
• Clear communication to explain data models to both technical and non-technical stakeholders.
• Bias toward shipping—iterate quickly rather than extended design cycles.

Responsibilities

• Design, build, and maintain scalable data pipelines in Databricks, ensuring data is reliable and well-documented.
• Own and evolve Databricks models, define schemas, enforce testing, and keep the warehouse clean.
• Automate manual data processes using Python, replacing spreadsheets with robust, repeatable jobs.
• Build dashboards and reports in Streamlit and HTMX (FastAPI) that Finance and Asset Management teams use.
• Integrate internal and third-party systems via REST APIs to improve data accessibility.
• Champion data quality with validation, monitoring, and governance practices.

Tools

Databricks

PostgreSQL

Streamlit

FastAPI

HTMX

GitHub

REST APIs

SQL

Python

Job description

Octopus Energy Group's OEGEN is seeking a Data Engineer to own data pipelines from source systems to decision-ready outputs, building pipelines in Databricks, modeling data, and delivering dashboards. You will automate processes with Python, integrate systems via REST APIs, and contribute to internal tools using Streamlit, FastAPI, or HTMX.

The role emphasizes data quality, collaboration with non-technical teams, and a hybrid work model in London, with a focus on scalable, auditable data