Databricks Data Engineer: Scalable Lakehouse Pipelines

Pulse Group; Reading; Sur place; GBP 63 000 - 77 000; Plein temps

Résumé du poste

Pulse Group partners with a major UK retailer to accelerate its data transformation. We seek a hands-on Data Engineer to design and implement scalable data pipelines using Databricks, Python, PySpark and SQL, within an Azure cloud environment.

You will collaborate with CRM, e-commerce and analytics teams to deliver reliable data platforms, embracing Lakehouse and Medallion architectures, and drive data quality and performance across production pipelines.

Qualifications

• Designing, building and maintaining scalable data pipelines using Databricks, Python, PySpark and SQL.
• Developing and optimising ETL/ELT processes to support analytics, reporting and digital products.
• Working with customer, CRM, e-commerce and operational datasets.
• Building reliable data solutions using modern Lakehouse and Medallion architecture principles.
• Integrating data from APIs, cloud platforms and multiple business systems.
• Working closely with stakeholders to understand requirements and translate them into technical solutions.
• Improving data quality, performance, reliability and engineering best practices.
• Contributing to a growing engineering function with plenty of opportunities to learn and take ownership.
• Commercial experience working with Databricks.
• Strong hands-on Python, PySpark and SQL skills.
• Experience developing production data pipelines and ETL/ELT processes.
• Understanding of modern data platforms, data modelling and Lakehouse architecture.
• Experience working within a cloud environment, ideally Azure.
• Strong problem-solving skills and the ability to communicate effectively with technical and non-technical stakeholders.
• Experience with Delta Lake, Medallion Architecture, Azure Data Factory or Unity Catalog.
• Exposure to dbt.
• Experience working with retail, e-commerce, CRM, marketing or customer data.
• An interest in AI-assisted engineering and emerging data technologies.

Responsabilités

• Designing, building and maintaining scalable data pipelines using Databricks, Python, PySpark and SQL
• Developing and optimising ETL/ELT processes to support analytics, reporting and digital products
• Working with customer, CRM, e-commerce and operational datasets
• Building reliable data solutions using modern Lakehouse and Medallion architecture principles
• Integrating data from APIs, cloud platforms and multiple business systems
• Working closely with stakeholders to understand requirements and translate them into technical solutions
• Improving data quality, performance, reliability and engineering best practices
• Contributing to a growing engineering function with plenty of opportunities to learn and take ownership
• Commercial experience working with Databricks
• Strong hands-on Python, PySpark and SQL skills
• Experience developing production data pipelines and ETL/ELT processes
• Understanding of modern data platforms, data modelling and Lakehouse architecture
• Experience working within a cloud environment, ideally Azure
• Strong problem-solving skills and the ability to communicate effectively with technical and non-technical stakeholders
• Experience with Delta Lake, Medallion Architecture, Azure Data Factory or Unity Catalog
• Exposure to dbt
• Experience working with retail, e-commerce, CRM, marketing or customer data
• An interest in AI-assisted engineering and emerging data technologies

Connaissances

Databricks

Python

PySpark

SQL

ETL/ELT

Data modelling

Lakehouse architecture

Stakeholder communication

Cloud data platforms

AI interest

Outils

Azure

Delta Lake

Medallion Architecture

Azure Data Factory

Unity Catalog

dbt

APIs integration

Description du poste

Pulse Group partners with a major UK retailer to accelerate its data transformation. We seek a hands-on Data Engineer to design and implement scalable data pipelines using Databricks, Python, PySpark and SQL, within an Azure cloud environment.

You will collaborate with CRM, e-commerce and analytics teams to deliver reliable data platforms, embracing Lakehouse and Medallion architectures, and drive data quality and performance across production pipelines.