Data Platform Engineer: Scale & Automate Data Pipelines

NewDay Technology; Greater London; On-site; GBP 70,000 - 90,000; Full time

Job summary

NewDay Technology is seeking a Data Engineer to design, build and evolve core pipelines powering analytics and data science. You will own the data lake and data platform, ensure reliable ETL/ELT, and drive scalable, cloud-native data solutions within an Agile framework.

You will optimise performance, collaborate with teams, and contribute to automation and best practices across the data stack.

Qualifications

• H hands-on experience with DBT and Apache Airflow.
• Experience solving complex data problems at scale with Python and SQL.
• Knowledge of data modelling, ETL/ELT, data warehousing and scalable pipelines.
• Ability to lead and mentor Data Engineers, providing technical direction.
• Interest in AI and automation to improve engineering quality and productivity.
• Comfort working in an Agile environment with cloud-native data solutions.

Responsibilities

• Developing and improving ingestion and transformation pipelines using Spark, Scala, Python, AWS, DBT, and Snowflake
• Owning and improving the organisation's Data Lake environment and data platform
• Owning ETL processing, table refreshes, defect resolution and other BAU activities
• Continuously optimising performance through concurrency, multi-threading, automation and best-practice patterns
• Working within Agile delivery to ship cloud-native data solutions via CI/CD
• Integrating data from multiple sources and solving sophisticated data challenges with creativity
• Design and develop modular, reusable scalable and configurable data engineering solutions adhering to DRY

Skills

Python

SQL

Data modelling

CI/CD

Leadership

Tools

DBT

Apache Airflow

AWS

Snowflake

Job description

NewDay Technology is seeking a Data Engineer to design, build and evolve core pipelines powering analytics and data science. You will own the data lake and data platform, ensure reliable ETL/ELT, and drive scalable, cloud-native data solutions within an Agile framework.

You will optimise performance, collaborate with teams, and contribute to automation and best practices across the data stack.