Data Engineer | Scalable Pipelines & Cloud Data

Leap29; Greater London; On-site; GBP 70,000 - 110,000; Full time

Job summary

Leap29 is seeking a Data Engineer to work on large-scale data projects, building reliable pipelines and scalable solutions to process and transform data from multiple sources. You will design, build and maintain pipelines, transform data across models, and develop reusable solutions using Python and best practices.

The role requires strong Python skills, experience with cloud platforms (Azure), Spark/Databricks, and data storage technologies such as Parquet/Delta Lake.

Qualifications

• Proven Data Engineering experience in large-scale environments.
• Strong Python skills and experience building data pipelines.
• Experience with data modelling and transforming data between formats.
• Knowledge of cloud platforms, ideally Azure, alongside Spark and Databricks.
• Familiarity with Parquet and Delta Lake storage formats.
• Experience with Docker, Kubernetes, Git/GitHub and Linux/Unix environments.

Responsibilities

• Design, build and maintain scalable data pipelines.
• Transform and standardise data across different data models.
• Develop reusable, maintainable solutions using Python and software engineering best practices.
• Work with cloud platforms, distributed processing technologies and modern data architectures.
• Collaborate with engineers and stakeholders in an Agile environment.

Skills

Python

Data pipelines

Cloud platforms (Azure)

Spark

Databricks

Parquet/Delta Lake

Docker

Kubernetes

Git/GitHub

Linux/Unix

Tools

Docker

Kubernetes

Git/GitHub

Linux/Unix

Azure

Job description

Leap29 is seeking a Data Engineer to work on large-scale data projects, building reliable pipelines and scalable solutions to process and transform data from multiple sources. You will design, build and maintain pipelines, transform data across models, and develop reusable solutions using Python and best practices.

The role requires strong Python skills, experience with cloud platforms (Azure), Spark/Databricks, and data storage technologies such as Parquet/Delta Lake.