Junior Databricks Engineer

Allscreens Nationwide Ltd; Greater London; On-site; GBP 28,000 - 50,000; Full time

Job summary

Allscreens Nationwide Ltd is seeking a Junior Databricks Engineer to join a client project in a hybrid/remote UK setup. The role focuses on building scalable data pipelines using Azure technologies, Delta Lake and Spark, under guidance from senior engineers.

You will work with ADLS Gen2, ADF, PySpark and Power BI integration, contributing to enterprise-scale data platforms and analytics.

Qualifications

• 2–5 years of experience in data engineering or related field.
• Experience with Azure Data Factory and PySpark in production.
• Familiarity with Azure Data Lake Storage Gen2 and Delta Lake.
• Experience building Lakehouse solutions with Apache Spark.
• Version control with Git; collaboration via Azure DevOps.

Responsibilities

• Design and implement ETL/ELT pipelines using Azure Data Factory.
• Write efficient SQL queries and transform data with Python/PySpark.
• Ingest, process, and store data in ADLS Gen2.
• Build and maintain Lakehouse solutions using Delta Lake and Spark.
• Monitor and optimise data pipelines for reliability and performance.
• Collaborate with senior engineers, analysts and stakeholders.
• Participate in code reviews and follow Git/Azure DevOps practices.
• Integrate datasets with Power BI and reporting tools.
• Document solutions and support continuous improvement.

Skills

SQL

Python

PySpark

ETL/ELT

Azure Data Factory

Tools

Azure Data Factory

Delta Lake

Apache Spark

ADLS Gen2

Git

Azure DevOps

Job description

About the Role

We are looking for a motivated **Junior Databricks Engineer** to join an exciting client project. This is an excellent opportunity for an aspiring Data Engineer to work with modern Azure technologies, Microsoft Fabric, and cloud-based data platforms while gaining hands-on experience in delivering enterprise-scale data solutions.

You will work alongside experienced Data Engineers and Solution Architects to develop, optimise, and support scalable data pipelines and analytics platforms.

---

Location

UK (Remote/Hybrid – Client Requirement)

Contract Type

Contract

Experience Required

2–5 Years

SC Clearance

Not Required

Key Responsibilities

• Support the design and implementation of ETL/ELT solutions using Azure Data Factory (ADF).
• Write efficient SQL queries and develop data transformation logic using Python and PySpark.
• Work with Azure Data Lake Storage (ADLS Gen2) to ingest, process, and store data.
• Assist in building and maintaining Lakehouse solutions using Delta Lake and Apache Spark.
• Monitor, troubleshoot, and optimise data pipelines to ensure reliability and performance.
• Collaborate with senior engineers, business analysts, and stakeholders to deliver high-quality data solutions.
• Participate in code reviews and follow best practices for version control using Git and Azure DevOps.
• Contribute to integrating datasets with Power BI and other reporting solutions.
• Document technical solutions and support continuous improvement initiatives.

Why Join Us?

• Work with the latest Azure, Databricks, and Microsoft Fabric technologies.
• Flexible remote/hybrid working arrangements.
• Opportunity to learn from experienced Azure Data Engineering professionals.
• Exposure to enterprise-scale cloud data platforms.
• Collaborative and supportive team environment.