Data Platform Engineer

Octopus Energy Group; City Of London; On-site; GBP 80,000 - 100,000; Full time

Job summary

A leading energy technology firm in London is seeking a Data Engineer to develop and maintain a data platform that drives smart energy usage. The ideal candidate should have strong skills in Python and AWS, and a passion for high-quality coding practices. You'll engage in projects aimed at automating data workflows and supporting the energy transition, all while enjoying a flexible and innovative work environment.

Benefits

Flexible salary discussions

Unique company culture

Opportunities for personal growth

Qualifications

• Passion for writing high quality code and building lean processes.
• Ability to work autonomously and proactively to scope problems and deliver solutions.

Responsibilities

• Build and optimise the platform to transform raw data into insights.
• Scope and deliver new data engineering projects with stakeholders.
• Develop and deploy AI tools for various use cases.
• Maintain and expand data infrastructure.

Skills

Python

AWS

SQL (Spark/Postgres preferable)

Tools

Docker

Spark

Airflow

Circle CI

DBT

Job description

Overview

At Octopus we've developed a data platform used by our businesses worldwide. The platform enables self-service data analytics and automates data processing workflows, from ETL to ML training and prediction. The data platform team works on initiatives from natural language understanding of customer communications to processing billions of smart meter readings daily to support customised and market-leading smart energy tariffs. We are looking for an Engineer to join the team to help build and maintain our platform, pipelines and reporting tools. We are passionate about building great technology to change how customers use energy for the good of the planet and this is a fantastic opportunity to work on data problems that move us closer to Net Zero and support the energy transition. We employ software engineering best practices to design, test, and deploy our data platform and services.

What you’ll do

• Build and optimise the platform used by the data team to transform raw data into business insights
• Scope and deliver new data engineering projects in collaboration with business stakeholders
• Develop and deploy AI tools to support growing AI requirements and use cases
• Research and implement new tooling, whether in-house or sourced externally
• Maintain and expand our existing data infrastructure and tools
• Consult the wider business on Engineering best practices and stay up to date with industry trends
• Support the internationalisation of our data infrastructure as we continue to grow globally

What you’ll have

• Primarily, a passion for writing high quality code and building lean processes

Essential:

• Python
• AWS
• SQL (Spark/Postgres preferable)

Areas for Growth:

• Airflow
• Experience using CLI tools
• Docker/EKS
• Spark
• DBT
• Circle CI

• The projects will be varied and we’re looking for someone who can work autonomously and proactively to scope problems and deliver pragmatic solutions

Our Tech Stack

• Python as our main programming language
• Circle CI for continuous deployment
• Databricks for our data lake platform
• Parquet and Delta formats on S3 for data lake storage
• Terraform on Spacelift for infrastructure definition and deployment
• Kubernetes for data services and task orchestration
• Airflow for job scheduling and tracking
• Postgres / Aurora for relational databases
• Spark for data processing
• dbt for data modelling
• SparkSQL for analytics
• Streamlit for data applications

Why you’ll love it here

• Salary is discussed on a call with a recruiter to match your experience and the correct level; there is flexibility and we focus on finding the right fit
• Octopus Energy Group has a unique culture with autonomy, co-owners, and projects that break new ground
• We were named best company to work for in 2022 and appear on lists for great places to work; we celebrate our culture and leadership
• Visit our UK perks hub – Octopus Employee Benefits

Our process usually takes up to 4 weeks, but we’ll flex around what works for you. You’ll chat with our recruitment team and your Recruiter will guide you through the stages. If you have questions, email [e-mail].

If this sounds like you, we’d love to hear from you.

Are you ready for a career with us? We aim to provide you with the tools and environment to unleash your potential. If you require accommodations or have a unique preference, let us know and we’ll tailor the interview process where possible.

Studies have shown that some groups of people, like women, are less likely to apply unless they meet 100% of the requirements. If you like one of our jobs, we encourage you to apply. We are an equal opportunity employer and do not discriminate on protected attributes. We commit to providing equal opportunities, an inclusive work environment, and fairness for everyone.