Data Engineer

NextGenEnergyJobs; Greater London; Hybrid; GBP 70,000 - 110,000; Full time

Job summary

NextGenEnergyJobs is seeking a data engineer to design, build, and maintain scalable data pipelines in Databricks while ensuring data reliability. You will own Databricks models, define schemas, test, and maintain a clean data warehouse.

The role also focuses on automating processes with Python and delivering useful dashboards for Finance and Asset Management. You will integrate systems via REST APIs, promote data quality through monitoring and governance, and collaborate with non-technical

Qualifications

• Proficient in SQL and Python for pipeline development, data modelling and automation.
• Strong understanding of data modelling, warehousing patterns and ETL/ELT design.
• Clear communication to explain data models to non-technical stakeholders.
• Bias toward shipping: deliver often and iterate.

Responsibilities

• Design, build, and maintain scalable data pipelines in Databricks, ensuring data is reliable and well-documented.
• Own and evolve Databricks models, define schemas, enforce testing, and keep the warehouse clean.
• Automate manual data processes using Python, replacing spreadsheets and ad-hoc workflows.
• Build dashboards and reports in Streamlit and HTMX (FastAPI) used by Finance and Asset Management teams.
• Integrate internal and third-party systems via REST APIs to improve data accessibility.
• Champion data quality with validation, monitoring, and governance practices.
• Collaborate with non-technical teams to translate problems into data solutions.

Skills

SQL

Python

Clear communication

Delivery mindset

Tools

Databricks

Git/GitHub

PostgreSQL

Job description

Help us make a big green dent in the universe.

Key Responsibilities

• Design, build, and maintain scalable data pipelines in Databricks, ensuring data is reliable, timely, and well-documented.
• Own and evolve our Databricks models - define schemas, enforce testing, and keep the warehouse clean.
• Automate manual data processes using Python, replacing spreadsheets and ad-hoc workflows with robust, repeatable jobs.
• Build performance dashboards and reports in Streamlit and HTMX (FastAPI) that Finance, Asset Management, and Fund Management actually use.
• Integrate internal and third-party systems via REST APIs, improving data accessibility across the business.
• Champion data quality - implement validation, monitoring, and governance practices that give stakeholders confidence in the numbers.
• Collaborate directly with non-technical teams to understand their problems and translate them into data solutions.

Requirements

• Strong SQL and Python skills, used daily for pipeline development, data modelling, and automation.
• Experience with modern data stack tooling: a cloud data platform (Databricks preferred), and version control (Git/GitHub).
• A solid understanding of data modelling, warehousing patterns, and ETL/ELT design.
• To be comfortable working with Postgres or similar relational databases.
• Clear communication - you can explain a data model to a fund manager and discuss trade-offs with an engineer.
• A bias toward shipping: you’d rather get something working and iterate than spend weeks in design.