Principal Data Engineer - Databricks & AI Platform Lead

Colibri Digital; United Kingdom; Remote; GBP 148,000 - 221,000; Full time

Benefits offered by this job

Remote-friendly

Training & certifications

Client engagement exposure

Job summary

Colibri Digital is seeking an experienced Principal Data Engineer on a contract basis to support complex client engagements across Data & AI transformation programmes. You will work directly with senior client stakeholders and delivery teams to design scalable data platforms and lead on Databricks initiatives.

This role requires deep Databricks expertise, architectural leadership and strong consulting skills to translate technology into business value.

Qualifications

• Proven experience as Principal Data Engineer, Lead Data Engineer or Data Architect within enterprise environments.
• Strong consultancy and client-facing experience across multiple customer engagements.
• Deep expertise in Databricks, including architecture, optimisation and leading delivery teams.
• Strong experience designing and delivering cloud-native data platforms within Azure and/or AWS environments.
• Advanced experience with PySpark, SparkSQL and large-scale distributed data processing.
• Strong understanding of modern data architecture, ETL/ELT frameworks and data modelling.
• Excellent SQL skills and experience working with large-scale datasets.
• Strong understanding of data governance, security and platform reliability principles.
• Excellent communication, stakeholder management and consulting skills.

Responsibilities

• Act as a trusted technical advisor across client engagements and build relationships with senior stakeholders.
• Lead the design and implementation of enterprise Databricks solutions and data platforms.
• Define engineering standards and scalable delivery approaches across cloud-native initiatives.
• Mentor engineering teams and drive DevOps, automation and incident resolution.
• Deliver high-quality, scalable data platforms and ensure governance and security.

Skills

Consultancy

Client-facing

Stakeholder management

Communication skills

Delivery-focused

Tools

Databricks

PySpark

SparkSQL

Azure

AWS

Job description

Colibri Digital is seeking an experienced Principal Data Engineer on a contract basis to support complex client engagements across Data & AI transformation programmes. You will work directly with senior client stakeholders and delivery teams to design scalable data platforms and lead on Databricks initiatives.

This role requires deep Databricks expertise, architectural leadership and strong consulting skills to translate technology into business value.