Senior Azure Databricks Data Engineer Lakehouse Analytics

Experis UK; Leeds; On-site; GBP 70,000 - 110,000; Full time

Job summary

Experis UK is seeking an experienced Senior Azure Databricks Data Engineer to join a growing team delivering large-scale cloud data transformation programmes across its Retail portfolio.

You will design, build and support scalable cloud data solutions using Azure technologies and Azure Databricks, working with Architects, Data Scientists, Analysts and Stakeholders to deliver trusted data products and reporting solutions.

Qualifications

• 8+ years of data engineering experience.
• Strong experience with Azure Databricks.
• Hands-on Python and PySpark development.
• Experience with Azure Synapse Analytics and ADLS.
• Strong SQL development and query optimisation.
• Experience building ETL/ELT frameworks.
• Data Warehousing and dimensional modelling experience.
• Experience with enterprise-scale cloud data platforms.
• Version control and CI/CD using Git and Azure DevOps.
• Strong stakeholder engagement and requirements gathering.
• Experience in Agile delivery environments.

Responsibilities

• Design, develop and maintain enterprise-scale Azure Databricks solutions.
• Build and optimise scalable ETL/ELT pipelines using Azure Data Factory and Databricks.
• Process and transform large datasets using Python, PySpark and Spark SQL.
• Develop batch and near real-time data ingestion solutions.
• Implement robust error handling, validation and monitoring frameworks.
• Design and implement modern Data Lake and Lakehouse architectures.
• Develop scalable dimensional data models supporting analytics and reporting.
• Build and maintain enterprise Data Warehouses and analytical data platforms.
• Contribute to architecture decisions and platform modernisation initiatives.
• Promote engineering best practices across performance, scalability and maintainability.
• Translate business requirements into scalable technical solutions.
• Support customer, product, sales, inventory and supply chain analytics initiatives.
• Work closely with stakeholders to deliver trusted data products and reporting solutions.
• Support self-service analytics and downstream BI environments.
• Implement CI/CD pipelines and Infrastructure-as-Code best practices.
• Manage source control using Git and Azure DevOps.
• Ensure data quality, governance and security standards are maintained.
• Produce technical documentation, architecture designs and support material.

Skills

Azure Databricks

Python / PySpark

Azure Synapse

Azure Data Lake

SQL development

ETL/ELT pipelines

Data Warehousing

Git & Azure DevOps

Agile delivery

Stakeholder engagement

Cloud data platforms

Tools

Git

Azure DevOps

Databricks

Spark SQL

Job description

Experis UK is seeking an experienced Senior Azure Databricks Data Engineer to join a growing team delivering large-scale cloud data transformation programmes across its Retail portfolio.

You will design, build and support scalable cloud data solutions using Azure technologies and Azure Databricks, working with Architects, Data Scientists, Analysts and Stakeholders to deliver trusted data products and reporting solutions.