Hybrid Data Engineer: Databricks, Fabric & Lakehouse

Leonard Curtis Recovery Limited; United Kingdom; Hybrid; GBP 60,000 - 95,000; Full time

Benefits offered by this job

25 days holiday

Birthday leave

Hybrid working

Flexible benefit and well-being

2 giving back days

Hard working, fun and professional env

Enhanced family friendly policies

Job summary

Leonard Curtis Recovery Limited is seeking a Data Engineer to design, build, and optimise robust data pipelines for enterprise analytics. The role emphasizes ETL/ELT workflows using Microsoft Fabric tools and the Databricks Lakehouse Platform to deliver clean, production-ready data.

You will implement Delta Live Tables, craft Medallion Architecture layers, and ensure governance via Unity Catalog. A strong focus on scalable, testable pipelines and CI/CD is essential.

Qualifications

• Strong proficiency in PySpark, Databricks SQL, and T-SQL
• Experience building production pipelines in Databricks workspaces and Fabric
• Familiar with Parquet, JSON, and CSV data formats

Responsibilities

• Construct scalable data ingestion pipelines using Fabric Data Factory, Dataflow Gen2, and Databricks Workflows.
• Implement Delta Live Tables to automate batch and real-time processing.
• Build and maintain the Medallion Architecture silver and gold layers across shared storage.
• Write optimized transformations in Fabric Notebooks and Databricks clusters using PySpark and SQL.
• Configure data access, masking, and governance within Unity Catalog.
• Optimize Lakehouse table performance with Z-Order indexing and file compaction.
• Deploy pipelines using CI/CD, Git integration, and Databricks Asset Bundles.
• Prepare finalized gold layer datasets for real-time Power BI Direct Lake reporting.

Skills

PySpark

Databricks SQL

T-SQL

Delta Lake

Databricks

Fabric Data Factory

Dataflow Gen2

Delta Live Tables

Notebooks

Unity Catalog

Z-Order indexing

CI/CD

Tools

Databricks Workspaces

Fabric

Job description

Leonard Curtis Recovery Limited is seeking a Data Engineer to design, build, and optimise robust data pipelines for enterprise analytics. The role emphasizes ETL/ELT workflows using Microsoft Fabric tools and the Databricks Lakehouse Platform to deliver clean, production-ready data.

You will implement Delta Live Tables, craft Medallion Architecture layers, and ensure governance via Unity Catalog. A strong focus on scalable, testable pipelines and CI/CD is essential.