Databricks Data Engineer: Azure Data Pipelines & AI-Ready

Dufrain Consulting Ltd; England; On-site; GBP 70,000 - 100,000; Full time

Job summary

Dufrain Consulting Ltd in the United Kingdom seeks an experienced Data Engineer to help clients transform data landscapes into scalable platforms. You’ll design end-to-end data solutions, collaborate with business and technical stakeholders, and ensure reliable delivery.

You'll apply Azure-based data engineering, leverage Databricks, PySpark, SQL, and build reusable pipelines while focusing on quality and scalability in fast-paced projects.

Qualifications

• Strong experience designing and delivering data solutions on Azure.
• Broad data engineering skillset with depth in Databricks.
• Proficient in Python and PySpark, including OO design and unit testing.
• Experience building metadata-driven pipelines instead of bespoke sources.
• Knowledge of lakehouse and medallion architecture, with dimensional modelling.

Responsibilities

• Develop good working relationships with clients on projects with both business and technical colleagues.
• Deliver end-to-end data engineering solutions in a collaborative team environment.
• Escalate issues to clients or senior team members when necessary and propose solutions.
• Support building the consulting practice through knowledge sharing and data services contributions.

Skills

Azure

Databricks

Python

PySpark

SQL

Metadata-driven pipelines

Lakehouse

Medallion architecture

Dimensional modelling

Job description

Dufrain Consulting Ltd in the United Kingdom seeks an experienced Data Engineer to help clients transform data landscapes into scalable platforms. You’ll design end-to-end data solutions, collaborate with business and technical stakeholders, and ensure reliable delivery.

You'll apply Azure-based data engineering, leverage Databricks, PySpark, SQL, and build reusable pipelines while focusing on quality and scalability in fast-paced projects.