Senior Azure Databricks Platform Engineer (Remote)

Allscreens Nationwide Ltd; Greater London; Hybrid; GBP 90,000 - 120,000; Part time

Job summary

A senior Databricks Engineer is sought by Allscreens Nationwide Ltd to design and deliver cloud-based data platforms in a UK-based remote/hybrid setting. You will build scalable data pipelines on Azure Databricks, implement Delta Lake architectures, and optimize Spark workloads for enterprise BI and analytics.

The role requires 5+ years of data engineering experience, mastery of Python, PySpark, SQL, and familiarity with Fabric and ADLS Gen2.

Qualifications

• 5+ years of experience in data engineering using Azure technologies.
• Experience delivering cloud-based data platforms for BI and analytics.
• Strong understanding of data warehousing and lakehouse concepts.

Responsibilities

• Design, develop, and maintain enterprise-scale data pipelines using Azure Databricks.
• Build ETL/ELT solutions with Azure Data Factory and Microsoft Fabric.
• Develop data processing with Python, PySpark, and SQL.
• Implement Delta Lake and Lakehouse architectures.
• Work with ADLS Gen2 to manage large-scale datasets.
• Optimize Spark workloads for performance and cost efficiency.
• Design data models to support reporting and analytics.
• Integrate datasets with Power BI for reporting and visualization.
• Implement CI/CD pipelines using Azure DevOps and Git.
• Collaborate with architects, analysts, and stakeholders; troubleshoot production issues.

Skills

Azure Databricks

PySpark

SQL

Python

Delta Lake

Lakehouse architecture

Spark performance tuning

Data modeling

Power BI integration

CI/CD

Tools

Azure Data Factory

Microsoft Fabric

ADLS Gen2

Azure DevOps

Git

Python tooling

Job description

A senior Databricks Engineer is sought by Allscreens Nationwide Ltd to design and deliver cloud-based data platforms in a UK-based remote/hybrid setting. You will build scalable data pipelines on Azure Databricks, implement Delta Lake architectures, and optimize Spark workloads for enterprise BI and analytics.

The role requires 5+ years of data engineering experience, mastery of Python, PySpark, SQL, and familiarity with Fabric and ADLS Gen2.