Data Engineer (Tech Lead/Staff)

NextGenEnergyJobs; Greater London; On-site; GBP 90,000 - 130,000; Full time

Job summary

Octopus Electric Vehicles is seeking a senior data platform engineer to own and optimize our data architecture, ensuring reliability and scalability. You will shape the team, coach members on best practices, and raise the technical bar while staying hands-on with data engineering.

You will build new data sources and pipelines, integrate with third-party systems via APIs, and collaborate with senior leadership to present data projects and insights that drive business decisions.

Qualifications

• Proven ability to lead and mentor high-performance data teams.
• Strong coding skills with a bias for quality and lean processes.
• Experience building robust pipelines from diverse data sources.

Responsibilities

• Own and optimise our data platform architecture focusing on reliability and scalability.
• Shape the direction of the growing data team and coach team members on best practices.
• Build data sources and pipelines, including API integrations and third-party systems.
• Collaborate with leadership to present data projects and insights.

Skills

Python

AWS

Spark

Distributed computing

Data pipelines

MLOps

Airflow

Parquet

SQL

No-SQL

Tools

Databricks

Terraform

dbt

Kubernetes

Job description

At Octopus Electric Vehicles we’ve built a data platform that provides data services to all areas of our business.

Key Responsibilities

• Own and optimise our platform architecture (improve reliability, scalability, maintainability, etc)
• Shape the direction of our growing team and coach team members on best practices.
• Shape engineering outputs and raising the technical bar of the organization without losing touch with hands-on engineering
• Interact with Senior Leadership to present your data projects back to them
• Build new data sources and data pipelines that deliver key data and insights to the business (this will include integration with third party systems and ingestion through APIs)
• Scope and deliver new data engineering projects in collaboration with business stakeholders
• Develop and deploy ML infrastructure to help build out our ever-growing AI requirements and use cases
• Collaborate closely with our dev teams to build seamless integrations between our back-end databases and our data platform

Requirements

• First and foremost, a passion for decarbonisation
• Ability to work with ambiguity and own problems end-to-end
• Experience managing/mentoring/coaching high performance data teams
• A passion for writing high quality code and building lean processes
• Experience with distributed data processing
• Experience with monitoring, testing, and data quality
• Experience building robust pipelines from diverse sources e.g. parquet, SQL & no-SQL databases, API endpoints
• It would be helpful to have experience/expertise/knowledge in the following:
• Python
• AWS
• Spark & distributed computing
• dbt
• Terraform
• Data Warehousing (we use Databricks but experience with other platforms is fine)
• MLOps implementation (desirable but not required)
• Airflow (or other orchestration tooling)