Databricks Data Engineer

Dufrain; England; On-site; GBP 70,000 - 100,000; Full time

Job summary

Dufrain Consulting Ltd in the United Kingdom seeks an experienced Data Engineer to help clients transform data landscapes into scalable platforms. You’ll design end-to-end data solutions, collaborate with business and technical stakeholders, and ensure reliable delivery.

You'll apply Azure-based data engineering, leverage Databricks, PySpark, SQL, and build reusable pipelines while focusing on quality and scalability in fast-paced projects.

Qualifications

• Strong experience designing and delivering data solutions on Azure.
• Broad data engineering skillset with depth in Databricks.
• Proficient in Python and PySpark, including OO design and unit testing.
• Experience building metadata-driven pipelines instead of bespoke sources.
• Knowledge of lakehouse and medallion architecture, with dimensional modelling.

Responsibilities

• Develop good working relationships with clients on projects with both business and technical colleagues.
• Deliver end-to-end data engineering solutions in a collaborative team environment.
• Escalate issues to clients or senior team members when necessary and propose solutions.
• Support building the consulting practice through knowledge sharing and data services contributions.

Skills

Azure

Databricks

Python

PySpark

SQL

Metadata-driven pipelines

Lakehouse

Medallion architecture

Dimensional modelling

Job description

At Dufrain, we help organisations turn complex data landscapes into scalable, high-performing platforms that drive better decisions and business outcomes. From building robust pipelines and reusable assets to enabling AI-ready architectures.

Data engineering is the heartbeat of every modern data platform, our engineers create the foundations that help clients unlock the full value of their data.

ROLE DETAILS

Our Data Engineers play a pivotal role in supporting clients as they navigate the complexities of data management, analytics, and strategy. Responsibilities include but are not limited to;

• Develop good working relationships with clients on a project including interpersonal skills with both business and technical focused colleagues.
• Experience working as a data engineer to develop performant end-to-end solutions in a collaborative team environment.
• Delivering high-quality pieces of work, proven ability to escalation problems to client / senior team members where necessary and propose possible solutions.
• Support building the Consulting practice through contribution to ongoing initiatives. This can include contributing to knowledge-sharing activities, and data services.

ABOUT YOUR & EXPERIENCE

You’re a driven, proactive experienced Data Engineer who thrives in fast‑paced environments and consistently delivers accurate, reliable solutions. You communicate with clarity, contribute confidently in meetings with senior stakeholders, and take ownership of your part in project delivery. Equally comfortable working independently or within an integrated squad, you support and share knowledge with others, stay current with industry standards, and meet tight deadlines while maintaining high engineering quality.

Essential Technical skills and experience:

• Strong experience designing and delivering data solutions on Azure
• Broad data engineering skillset, with depth in Databricks
• Strong Python / PySpark, including object-oriented design, building reusable libraries and packages, and unit testing, alongside strong SQL
• Experience building and maintaining metadata-driven or parameterised pipelines rather than hand-built one-per-source solutions
• Solid grasp of lakehouse and medallion architecture, dimensional modelling

Why join Dufrain?

Join a people‑first consultancy where collaboration, learning and growth are part of everyday life. You'll work on diverse projects across industries, helping organisations turn data into insight and insight into action, while developing your skills alongside experienced data and AI professionals.