Azure Databricks Engineer - Hybrid, London

Opus Recruitment Solutions Limited; Greater London; Hybrid; GBP 92,000 - 101,000; Full time

Job summary

Opus Recruitment Solutions Limited in London is seeking an experienced Azure Databricks Engineer to design, develop and support scalable data solutions across Azure and Databricks. The role combines hands-on engineering with collaboration across the data team to deliver reliable, high-quality data for business insights.

You will work on pipelines, Delta Lake, Unity Catalog and modern data lake architectures, alongside CI/CD and automation practices.

Qualifications

• Strong hands-on experience with Azure Databricks in an enterprise environment.
• Excellent SQL Server/T-SQL and data engineering skills.
• Experience building and optimising large-scale data pipelines.
• Knowledge of Spark, Delta Lake, Unity Catalog and Azure data services.
• Exposure to Git, CI/CD and infrastructure automation tools.
• Strong communication skills and a proactive approach to problem solving.

Responsibilities

• Develop Databricks solutions using SQL, PySpark and Python.
• Build and maintain ETL/ELT pipelines across Azure, SQL Server and Databricks.
• Support Lakeflow pipelines, production jobs and data quality processes.
• Monitor, troubleshoot and optimise data workflows and platform performance.
• Work with Delta Lake, Unity Catalog and modern data lake architectures.
• Integrate Azure services including ADLS, Data Factory, Key Vault and REST APIs.
• Support CI/CD deployments and Databricks automation.

Skills

Azure Databricks

SQL Server/T-SQL

Data pipelines

Spark

Delta Lake

Unity Catalog

Azure data services

Git

CI/CD

Automation tools

Job description

Opus Recruitment Solutions Limited in London is seeking an experienced Azure Databricks Engineer to design, develop and support scalable data solutions across Azure and Databricks. The role combines hands-on engineering with collaboration across the data team to deliver reliable, high-quality data for business insights.

You will work on pipelines, Delta Lake, Unity Catalog and modern data lake architectures, alongside CI/CD and automation practices.