Senior Data Platform Engineer - AWS & Databricks

NextGenEnergyJobs; Greater London; Hybrid; GBP 65,000 - 90,000; Full time

Job summary

NextGenEnergyJobs is seeking a full-time data engineer to establish the flexibility team's data capability and set its technical direction within an AWS-based environment. You will upskill data-literate backend engineers, act as the team's data expert, and drive scalable, reusable data pipelines and APIs using Django/FastAPI where appropriate.

You'll maintain and evolve the data warehouse with Databricks, champion DataOps practices, and ensure robust, globally scalable data applications across

Qualifications

• Proficient in SQL, Python and AWS data stack.
• Experience with Databricks data warehouse architecture.
• Familiar with Kubernetes, Docker, Spark and DataOps monitoring tools.

Responsibilities

• Establish the data capability of the flexibility team and set its technical direction.
• Upskill backend engineers and multiply their impact.
• Build greenfield data systems in an AWS environment.
• Provide a robust data processing and warehousing layer for current and future projects.
• Maintain and improve data applications for global scalability and operational efficiency.
• Collaborate with the data warehouse built on Databricks and drive platform stability.

Skills

SQL

Python

AWS

Django

FastAPI

Kubernetes

Docker

Spark

DataDog

Grafana

Prometheus

Airflow

dbt

Tools

Databricks

Job description

NextGenEnergyJobs is seeking a full-time data engineer to establish the flexibility team's data capability and set its technical direction within an AWS-based environment. You will upskill data-literate backend engineers, act as the team's data expert, and drive scalable, reusable data pipelines and APIs using Django/FastAPI where appropriate.

You'll maintain and evolve the data warehouse with Databricks, champion DataOps practices, and ensure robust, globally scalable data applications across