Senior Data Engineer

So Energy; Greater London; Hybrid; GBP 90,000 - 130,000; Full time

Benefits offered by this job

Hybrid Working

Learning Platform

Educational Sponsorships

Real Living Wage Employer

Birthday Off

Three So Giving Days

Enhanced Family Leave

Employee Assistance Programme

Job summary

So Energy in London (Chiswick) is seeking a Senior Data Engineer to design, build and scale data pipelines on the Google Cloud Platform. You will own hands‑on implementation, optimize pipelines, and mentor junior engineers while advancing our lakehouse and data governance.

You’ll work in a hybrid setup with cross‑functional partners, apply best practices, and contribute to high‑impact analytics projects that power sustainable energy solutions.

Qualifications

• Extensive experience building data pipelines on GCP with BigQuery, Dataflow and Cloud Storage.
• Strong SQL expertise for complex transformations and optimization.
• Proficient in Python for scalable data workflows and orchestration.
• IaC with Terraform to automate GCP resources and environments.
• Experience with CI/CD and containerization (Docker/Kubernetes) is essential.
• Understanding of data governance, IAM and security best practices.

Responsibilities

• Design, develop, and maintain scalable data pipelines on GCP.
• Build and optimize a lakehouse architecture and data models.
• Implement ingestion, transformation, and loading from diverse sources.
• Ensure data quality, security, and cost efficiency.
• Collaborate with analysts to support analytics solutions.
• Mentor junior data engineers and contribute to best practices.

Skills

GCP stack (BigQuery, Dataflow, Pub/Sub

SQL

Python

Terraform (IaC)

CI/CD pipelines

Docker

Kubernetes

Data modeling (star schema)

Data governance & IAM

Batch & streaming data pipelines

Tools

Docker

Kubernetes

Terraform

Job description

Our priority is to match the right person to the right role, and we never want salary to be a barrier to applying. Regardless of your current pay, we’ll ask about your salary expectations during the application process, and our Talent Team will have an open, honest conversation during the talent call to understand your skills and explore our salary options. If you have questions around salary, please feel free to ask us!

Location: Chiswick, London - Hybrid

Sponsorship: We are unable to offer sponsorship for this role

The Role

As a Senior Data Engineer, you will play a key role in the development and maintenance of our data infrastructure, ensuring efficient and scalable data pipelines.

You will be responsible for hands‑on implementation and optimization of data systems on the Google Cloud Platform (GCP), contributing to critical projects while mentoring junior Engineers.

This role focuses on delivering technical solutions, applying best practices, and driving innovation within the data engineering team.

Why So Energy?

So Energy was created in 2015 because we knew energy suppliers could be better. Since then, we’ve grown rapidly but sustainably, with 300,000 customers and over 450 Energists (what we call our people). But we’re not done! We’re on the road to a net‑zero future, and thanks to our partnership with ESB, we’re well on the way. We’re customer‑centric, tech‑led, and passionate about sustainability.

We’re driven to do our best for our customers, for each other, and for our planet. That’s why we’ve built a workplace culture that’s supportive, empowering, inclusive, and full of opportunities to grow and make an impact.

At So Energy, we take pride in our values‑driven culture. We live and breathe by our six core values that guide everything we do:

• Clear

• Honest

• Ambitious

• Inquisitive

• Caring

• Sustainable

What you’ll be getting up to in this role:

• Design, develop, and maintain scalable data pipelines using modern data engineering tools and technologies on our GCP stack.

• Build and optimize our lake house on Google Cloud Platform (GCP).

• Implement data ingestion, transformation, and loading processes for various data sources (e.g., databases, APIs, cloud storage).

• Ensure data quality, consistency, and security throughout the data pipeline.

• Leverage GCP services (e.g., Dataflow, Dataproc, BigQuery, Cloud Storage) to build and maintain cloud‑native data solutions.

• Implement infrastructure as code (IaC) principles using Terraform to automate provisioning and configuration.

• Manage and optimize cloud resources to ensure cost‑efficiency and performance.

• Design and implement efficient data models following a star schema approach to support analytical and operational workloads.

• Collaborate with data analysts to develop advanced analytics solutions.

• Conduct data quality analysis to drive better data management on outputs in our Curated Layer.

• Mentor junior data engineers and provide technical guidance.

• Contribute to the development of data engineering best practices and standards.

• Collaborate with cross‑functional teams to deliver complex data projects.

You’re a great match if:

Technical Skills

• You're an expert in GCP services including BigQuery, Dataflow, Pub/Sub, Cloud Composer, Cloud Storage, and Cloud Functions. GCP Professional Data Engineer Certification is highly favourable.

• Advanced knowledge of SQL for complex data transformation and query optimization.

• Proven experience in Python for scalable data pipeline development and orchestration following best practices.

• Experience implementing Terraform for Infrastructure as Code (IaC) to automate GCP resource management.

• Knowledge of CI/CD pipelines and automated deployment practices.

• Experience with containerization technologies (e.g., Docker, Kubernetes)

• Experience building and optimizing batch and streaming data pipelines.

• Understanding of data governance principles, GCP security (IAM, VPC), and compliance requirements.

Soft Skills

• Demonstrates a growth mindset by actively seeking to learn from peers and stakeholders, fostering a culture of open communication and shared knowledge.

• Works effectively across teams, including Data Science, Engineering, and Analytics, to understand their needs and deliver impactful data solutions.

• Actively participates in design discussions, brainstorming sessions, and cross‑functional projects, always striving for continuous improvement and innovation.

• Builds strong relationships across the organization, using empathy and active listening to ensure alignment on goals and deliverables.

• Approaches challenges with a growth mindset, viewing obstacles as opportunities to innovate and improve processes.

• Applies a structured and analytical approach to solving complex problems, balancing immediate needs with long‑term scalability and efficiency.

• Demonstrates resilience under pressure, maintaining a positive and solution‑focused attitude when faced with tight deadlines or ambiguity.

• Actively seeks feedback and lessons learned from past projects to continuously refine problem‑solving strategies and improve outcomes.

• Shares expertise generously, guiding team members in adopting best practices and helping them overcome technical challenges.

• Leads by example, demonstrating how to approach complex problems pragmatically while promoting curiosity and a willingness to explore new tools and technologies.

• Encourages professional development within the team, supporting individuals in achieving their career goals and obtaining certifications, especially within the Google Cloud ecosystem.

Research shows that some underrepresented people are less likely to apply for a role unless they are 100% qualified. We believe your experience, skills, and passion will set you apart. We encourage you to apply and tell us about your journey, even if you don’t tick all the boxes.

What’s in it for you?

Smart Working Charters:

• Hybrid Working – Every team has its own SPARK Working Charter. Ask your Talent Partner for more details.

Growth & Development

• Dedicated Learning & Development Team – Supporting your development through workshops, coaching and learning opportunities.

• Learning Platform – Access thousands of learning resources to help you build new skills and support your career development.

• Educational Sponsorships – Supporting professional development through initiatives such as Code First Girls.

Pay & Reward

• Up to 10% Performance Bonus – Based on both company and personal performance.

• Annual Salary Reviews – Helping ensure we remain competitive in the market.

• Real Living Wage Employer – Proud to be an accredited Real Living Wage employer.

Time Off That Matters

• Your Birthday Off – Because birthdays should be celebrated.

• Three So Giving Days – Paid time to support a cause that's important to you.

• Enhanced Family Leave – Supporting you through every stage of family life.

Health & Wellbeing

• Employee Assistance Programme (EAP) – Confidential support available 24/7, 365 days a year, including access to professional guidance and structured counselling support.

• Wellbeing Resources via Perkbox – Access a range of wellbeing articles, tools and guidance, alongside employee perks and discounts, all in one place.

• Physical Wellbeing – Free eye tests and annual flu vaccinations.

• Menopause Policy – Providing support and guidance for colleagues experiencing menopause.

Belonging & Recognition

• Affinity Groups – Connect with colleagues through our employee‑led communities.

• Monthly Social Events – Bringing teams together and creating opportunities to connect.

• Charitable Fundraising – Supporting the communities we're proud to be part of.

Diversity, Equity, Inclusion & Belonging

As a Disability Confident Committed employer, we encourage applications from everyone, and we will ensure fair and accessible recruitment for all. At So Energy, we’re committed to cultivating an environment that promotes diversity, equity, inclusion, and belonging. We are a global community, and we believe our unique qualities should be celebrated as they are critical to our innovation. It’s essential to us that you bring your authentic self to work every single day, no matter your age, ethnicity, religion, citizenship, gender identity, sexual orientation, disability status, caring responsibilities, neurodiversity, or otherwise. Inclusion isn’t just an initiative at So Energy. We strive to embed it throughout our entire culture.