Senior Data Platform Engineer – Lead Pipelines

NewDay; Greater London; Hybrid; GBP 90,000 - 130,000; Full time

Job summary

NewDay is seeking a Senior Data Engineer in the United Kingdom to lead the design, build, and evolution of core pipelines powering analytics, reporting, and data science across the business.

You will drive ingestion and transformation with Spark, Python, SQL, and AWS, shape the Data Lake, own ETL processes, push automation, and optimize performance while mentoring engineers and delivering cloud-native solutions via CI/CD, influencing governance and standards.

Qualifications

• Hands-on experience using DBT to design and build data transformations within a complex environment.
• Engineering experience using Python and SQL, with the ability to solve complex data problems at scale.
• Ability to design and guide modern data engineering solutions, with knowledge of data modelling, ETL/ELT, data warehousing and scalable data pipelines.
• Ability to lead and develop Data Engineers, combining hands-on technical contribution with coaching, technical direction and ownership of delivery.
• An interest in AI and automation, with the technical curiosity to explore how emerging technologies could improve engineering quality, productivity and ways of working.

Responsibilities

• Lead development and optimisation of ingestion and transformation pipelines using Spark, Scala, Python, AWS, DBT, and Snowflake.
• Architecting and evolving the organisation’s Data Lake environment to ensure it remains robust, scalable, secure, and future‑ready
• Owning and improving ETL processing, table refreshes, defect resolution, and other BAU activities, while driving automation to reduce manual effort
• Championing performance optimisation through concurrency, multi‑threading, automation, and best‑practice engineering patterns
• Guiding Agile delivery, mentoring engineers, and ensuring high‑quality, cloud‑native data solutions are shipped via CI/CD
• Integrating data from multiple sources and solving sophisticated data challenges with creativity, technical depth, and strategic thinking
• Influencing data engineering standards, patterns, and governance across the wider organisation

Skills

DBT

Python

SQL

Data modeling

ETL/ELT

Data warehousing

CI/CD

Leadership

Tools

Snowflake

Spark

Airflow

AWS

Scala

GitHub Actions

Job description

NewDay is seeking a Senior Data Engineer in the United Kingdom to lead the design, build, and evolution of core pipelines powering analytics, reporting, and data science across the business.

You will drive ingestion and transformation with Spark, Python, SQL, and AWS, shape the Data Lake, own ETL processes, push automation, and optimize performance while mentoring engineers and delivering cloud-native solutions via CI/CD, influencing governance and standards.