About the role
You will be part of our Data Platform team, which sits within Coop’s AI & Business Intelligence organisation – the team building the data foundation that powers analytics, reporting, operational intelligence and AI across the company.
In this role you will design, build and optimise enterprise-grade data platforms using Azure Data Services, Databricks, modern Data Lakehouse architectures, real‑time streaming and DevOps automation. You’ll turn data from across the business into trusted, high‑quality data products that the whole organisation can rely on.
What you will do
- Design and implement scalable batch and real‑time data pipelines using Azure‑native technologies.
- Build and maintain robust data ingestion frameworks from multiple source systems, APIs and event streams.
- Develop and manage Medallion Architecture (Bronze, Silver, Gold) within the enterprise Data Lake.
- Implement streaming solutions with Kafka, Event Hub and Service Bus for event‑driven processing.
- Develop transformation frameworks using Databricks, PySpark and Delta Lake.
- Collaborate with business stakeholders, data analysts and data scientists to deliver trusted, high‑quality data products.
- Enable CI/CD pipelines and automate deployments using Azure DevOps best practices.
- Drive platform optimisation focused on performance, scalability, reliability and cost efficiency.
- Ensure adherence to data governance, security, monitoring and operational excellence standards.
Who are you?
To succeed in this role, we believe you have a genuine passion for getting value from massive datasets and love working with state‑of‑the‑art technologies that make completely new data‑driven solutions possible. You enjoy working with different tasks and stakeholders, you are result‑oriented, and you spark when given room to take initiative and create solutions. As a team, we believe in supporting and challenging each other and growing a positive, learning culture.
We believe that you
- Have strong hands‑on experience with the Azure data platform – Azure Data Factory, Azure Databricks, Data Lake Storage Gen2 (ADLS), Azure Functions, Event Hub and Service Bus.
- Are highly skilled in Databricks and the Lakehouse – PySpark, Spark SQL, Delta Lake, Databricks Workflows, Unity Catalog and performance tuning, applying Medallion and Lakehouse architecture in practice.
- Are strong in data architecture and modelling – Lakehouse and Medallion design, ETL/ELT frameworks, and data quality & validation frameworks.
- Have experience with streaming and integration – Apache Kafka, event‑driven architecture, Service Bus, batch ingestion / Autoloader and REST APIs.
- Are a confident programmer in Python, PySpark and SQL.
- Work the DevOps way – Azure DevOps, CI/CD pipeline development and Infrastructure as Code (Terraform preferred).
- Are comfortable with monitoring and operations – Log Analytics, Application Insights, and performance tuning & troubleshooting.
- Can communicate effectively with both technical specialists and non‑technical stakeholders.
It’s meritorious if you
- Have 5+ years of experience in data engineering.
- Have built enterprise‑scale Azure data platforms hands‑on.
- Have worked with retail, logistics or large‑scale operational data environments.
- Have supported machine learning and advanced analytics workloads.
- Have a strong understanding of distributed data processing and streaming architectures.
Welcome with your application!