Senior Data Engineer

NewDay Technology; Greater London; Hybrid; GBP 90,000 - 120,000; Full time

Job summary

NewDay Technology in London is seeking a Senior Data Engineer to lead the design, build, and evolution of core data pipelines and platforms powering analytics, reporting, and data science across the business.

You will own ingestion and transformation pipelines using Spark, Python, SQL, AWS, DBT, and Snowflake, shape the Data Lake, drive CI/CD for cloud-native solutions, mentor engineers, and collaborate across teams in a hybrid working model.

Qualifications

• DBT experience designing and improving data transformations in a complex environment.
• Proficiency in Python and SQL with ability to solve large-scale data problems.
• Experience with data modelling, ETL/ELT, data warehousing and scalable pipelines.
• Ability to design and guide modern data engineering solutions and lead other engineers.
• Interest in AI and automation to improve engineering quality and productivity.

Responsibilities

• Lead development and optimisation of ingestion and transformation pipelines using Spark, Scala, Python, AWS, DBT and Snowflake.
• Architect and evolve the Data Lake to be robust, scalable, secure and future‑ready.
• Own ETL processing, table refreshes and BAU, driving automation to reduce manual work.
• Drive performance optimisation through concurrency, multi‑threading and best practices.
• Guide Agile delivery, mentor engineers, and ensure cloud‑native solutions are shipped via CI/CD.
• Integrate data from multiple sources and tackle advanced data challenges with strategic thinking.

Skills

DBT

Python

SQL

Data modelling

ETL/ELT

Data warehousing

Pipelines

Leadership

AI & automation

Cloud platforms

Tools

Snowflake

Spark

Scala

Airflow

AWS

GitHub

CI/CD

Job description

Are you ready to shape the backbone of a data‑driven organisation? NewDay are looking for a Senior Data Engineer to lead the design, build, and evolution of the core pipelines and platforms that power analytics, reporting, and sophisticated data science across our business.

How you'll contribute

• Leading the development and optimisation of ingestion and transformation pipelines using technologies such as Spark, Scala, Python, AWS, DBT, and Snowflake
• Architecting and evolving the organisation's Data Lake environment to ensure it remains robust, scalable, secure, and future‑ready
• Owning and improving ETL processing, table refreshes, defect resolution, and other BAU activities, while driving automation to reduce manual effort
• Championing performance optimisation through concurrency, multi‑threading, automation, and best‑practice engineering patterns
• Guiding Agile delivery, mentoring engineers, and ensuring high‑quality, cloud‑native data solutions are shipped via CI/CD
• Integrating data from multiple sources and solving sophisticated data challenges with creativity, technical depth, and strategic thinking
• Influencing data engineering standards, patterns, and governance across the wider organisation

We're looking for these essential skills

• Hands‑on experience using DBT to design, build and improve data transformations within a complex data environment.
• Engineering experience using Python and SQL, with the ability to solve complex data problems at scale.
• The ability to design and guide modern data engineering solutions, with knowledge of data modelling, ETL/ELT, data warehousing and scalable data pipelines.
• The ability to lead and develop Data Engineers, combining hands‑on technical contribution with coaching, technical direction and ownership of delivery.
• An interest in AI and automation, with the technical curiosity to explore how emerging technologies could improve engineering quality, productivity and ways of working.

It's a plus if you also have these skills

• Experience using AWS or another major cloud platform. Our environment includes EMR, Athena, Lambda, S3, Glue and Step Functions, but we're open to transferable experience solving similar problems.
• Experience with Snowflake, Spark, Scala or Airflow, or transferable experience with comparable technologies and solving similar data engineering challenges.
• Experience with Git, GitHub and CI/CD practices, including tools such as GitHub Actions.
• Knowledge of real‑time data processing, data governance, data quality or observability.
• Experience using AI‑enabled engineering tools or automation to improve data engineering productivity, quality or reliability.

We know great candidates don't always meet every requirement. If you have strong data engineering foundations, hands‑on DBT experience and experience solving sophisticated problems at scale, we'd encourage you to apply even if you haven't worked with every technology listed above.

The impact you'll make

Your work will strengthen the foundation of our entire data landscape. By improving reliability, reducing manual intervention, and accelerating time to insight, you'll unlock new opportunities for analytics, innovation, and strategic growth. You'll help define how data engineering is done at NewDay - and ensure the organisation is ready for what comes next.

#LI-Hybrid

#LI-TM1

At NewDay, we value all types of diversity. We're an equal opportunity employer and believe that our differences create a vibrant, authentic working culture. We want all our colleagues to feel able to bring their whole selves to work. We don't discriminate on the basis of protected characteristics or identities. We make sure that every job is crafted to be inclusive and that people with disabilities or caring responsibilities can take part in the application and interview process.

Tell us if you need accommodations: We'll put reasonable adjustments in place to support you.

We work with Textio to make our job design and hiring inclusive.

Permanent