Databricks Data Engineer: Azure Pipelines for AI-Ready

Dufrain Consulting Ltd; England; On-site; GBP 55,000 - 90,000; Full time

Job summary

Dufrain Consulting Ltd is seeking a driven Data Engineer to join our fast-paced consultancy. You will design and deliver scalable data solutions, collaborate with clients and cross-functional teams, and contribute to AI-ready architectures.

We value strong Python, PySpark, SQL skills, Databricks, and experience with lakehouse and medallion patterns, ensuring high-quality data products and reliable performance.

Qualifications

• Experience designing and delivering data solutions on Azure.
• Broad data engineering skills with Databricks, Python, PySpark, and SQL.
• Experience building metadata-driven or parameterised pipelines.
• Knowledge of lakehouse and medallion architecture and dimensional modelling.

Responsibilities

• Develop good working relationships with clients on projects, interacting with both business and technical colleagues.
• Deliver end-to-end data engineering solutions in a collaborative team environment.
• Produce high-quality work, escalate issues to clients/senior team members and propose solutions.
• Support building the Consulting practice through knowledge sharing and data services.

Skills

Azure

Databricks

Python

PySpark

SQL

Pipelines design

Lakehouse architecture

Dimensional modelling

Job description

Dufrain Consulting Ltd is seeking a driven Data Engineer to join our fast-paced consultancy. You will design and deliver scalable data solutions, collaborate with clients and cross-functional teams, and contribute to AI-ready architectures.

We value strong Python, PySpark, SQL skills, Databricks, and experience with lakehouse and medallion patterns, ensuring high-quality data products and reliable performance.