Data Engineer - Scalable Pipelines & Impactful Dashboards

NextGenEnergyJobs; Greater London; Hybrid; GBP 70,000 - 110,000; Full time

Job summary

NextGenEnergyJobs is seeking a data engineer to design, build, and maintain scalable data pipelines in Databricks while ensuring data reliability. You will own Databricks models, define schemas, test, and maintain a clean data warehouse.

The role also focuses on automating processes with Python and delivering useful dashboards for Finance and Asset Management. You will integrate systems via REST APIs, promote data quality through monitoring and governance, and collaborate with non-technical

Qualifications

• Proficient in SQL and Python for pipeline development, data modelling and automation.
• Strong understanding of data modelling, warehousing patterns and ETL/ELT design.
• Clear communication to explain data models to non-technical stakeholders.
• Bias toward shipping: deliver often and iterate.

Responsibilities

• Design, build, and maintain scalable data pipelines in Databricks, ensuring data is reliable and well-documented.
• Own and evolve Databricks models, define schemas, enforce testing, and keep the warehouse clean.
• Automate manual data processes using Python, replacing spreadsheets and ad-hoc workflows.
• Build dashboards and reports in Streamlit and HTMX (FastAPI) used by Finance and Asset Management teams.
• Integrate internal and third-party systems via REST APIs to improve data accessibility.
• Champion data quality with validation, monitoring, and governance practices.
• Collaborate with non-technical teams to translate problems into data solutions.

Skills

SQL

Python

Clear communication

Delivery mindset

Tools

Databricks

Git/GitHub

PostgreSQL

Job description

NextGenEnergyJobs is seeking a data engineer to design, build, and maintain scalable data pipelines in Databricks while ensuring data reliability. You will own Databricks models, define schemas, test, and maintain a clean data warehouse.

The role also focuses on automating processes with Python and delivering useful dashboards for Finance and Asset Management. You will integrate systems via REST APIs, promote data quality through monitoring and governance, and collaborate with non-technical