Principal Data Engineer

Colibri Digital

United States

Remote

GBP 100,000 - 150,000

Full time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Private health insurance
Remote-first work model
Certification access
Global client exposure

Job summary

Colibri Digital seeks a Principal Data Engineer to bridge complex data architectures and strategic business value. You will serve as the primary technical authority for clients, ensuring Databricks data platforms are scalable, resilient, and aligned with long-term business goals.

You will drive solution architecture, mentor teams, and lead multi-year roadmaps across Azure/AWS, PySpark, SparkSQL, and data governance, while engaging stakeholders with clarity and impact.

Qualifications

  • Proven experience in a principal engineering role.
  • Extensive experience with cloud data platforms (Azure or AWS) focusing on Databricks.
  • Strong Python (PySpark) and SparkSQL skills.
  • Strong data modelling concepts relational and non-relational databases.
  • Proficiency in SQL and experience with data integration and ETL tools.
  • Excellent knowledge of big data technologies and frameworks.
  • Strong knowledge of data governance and data security best practices.
  • Remote-first with client-facing responsibilities.
  • Excellent communication and stakeholder management skills.

Responsibilities

  • Act as the primary strategic partner to client business owners, aligning technical data solutions with corporate goals while championing reliability and "clean data" for all consumers.
  • Drive the development of complex, large-scale data architectures and multi-year technology roadmaps, balancing technical innovation with cost efficiency and operational stability.
  • Oversee delivery performance, ensuring the quality and resilience of high-impact data pipelines while managing capacity planning and team velocity.
  • Lead the triage and resolution of critical platform incidents, taking full ownership of root-cause prevention and maintaining transparent communication with clients during recovery.
  • Implement robust governance frameworks and compliance controls, acting as the authoritative lead on risk mitigation and the primary technical contact for regulatory matters.
  • Champion an organisational culture of knowledge sharing, representing the firm at industry events and establishing best practices across communities of practice.
  • Influence long-term organizational health by driving platform efficiency, managing engineering budgets, and leading strategic hiring and retention initiatives.

Skills

Leadership
Solution architecture
Stakeholder management
Python
Data modelling
Data governance
Communication
Remote work

Tools

Databricks
Azure
AWS
PySpark
SparkSQL
SQL
ETL tools

Job description

As a Principal Data Engineer, you will serve as the bridge between complex technical architecture and strategic business value. This is a high-impact, client-facing leadership role designed for an expert—ideally holding a Databricks Champion or Databricks Professional certification—who can navigate deep technical challenges while maintaining a "customer-first" mindset. You will act as the primary technical authority for our clients, ensuring their Databricks data platforms are resilient, scalable, and aligned with their long-term business goals.

Key Responsibilities
  • Customer Excellence & Stakeholder Management: Act as the primary strategic partner to client business owners, aligning technical data solutions with corporate goals while championing reliability and "clean data" for all consumers.
  • Solution Architecture & Tech Strategy: Drive the development of complex, large-scale data architectures and multi-year technology roadmaps, balancing technical innovation with cost efficiency and operational stability.
  • Delivery & Execution: Oversee "team-of-teams" delivery performance, ensuring the quality and resilience of high-impact data pipelines while managing capacity planning and team velocity.
  • Incident Management & Resilience: Lead the triage and resolution of critical platform incidents, taking full ownership of root-cause prevention and maintaining transparent communication with clients during recovery.
  • Governance, Risk & Compliance: Implement robust governance frameworks and compliance controls, acting as the authoritative lead on risk mitigation and the primary technical contact for regulatory matters.
  • Knowledge Management & Mentorship: Champion an organisational culture of knowledge sharing, representing the firm at industry events and establishing best practices across communities of practice.
  • Business Strategy: Influence long-term organizational health by driving platform efficiency, managing engineering budgets, and leading strategic hiring and retention initiatives.
Skills, Knowledge and Expertise
  • Proven experience in a principal engineering role.
  • Extensive experience with cloud data platforms such as Azure or AWS, with a focus on Databricks.
  • Highly skilled with Pythonic data processing tools such as Pyspark and SparkSQL.
  • Strong understanding of data modelling concepts relational and non-relational databases.
  • Proficiency in SQL and experience with data integration and ETL tools.
  • Excellent knowledge of big data technologies and frameworks.
  • Strong knowledge of data governance and data security best practices.
  • Strong analytical mindset with attention to detail and a consultative approach.
  • Excellent communication and stakeholder management skills.
  • Remote-first working with flexibility around client needs.
  • Private health insurance.
  • Projects with global brands & public sector leaders, working on real-world impact.
  • Exclusive access to AWS & Databricks certifications, training, and partner tools.
  • A collaborative culture of technical excellence and continuous learning.
  • A lead role in shaping delivery standards and future Data & Consulting practices within the firm.
  • Competitive salary, bonus structure, and long-term growth opportunities.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Solution Architect, Delivery (Databricks)
Solution Architect, Delivery (Databricks)

Jobgether • United States

Remote
USD 140,000 - 210,000
Remote work within the United States
Competitive compensation
Professional development
+1
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

UNAVAILABLE • McLean (VA)

On-site
USD 120,000 - 160,000
Senior Databricks Engineer
Senior Databricks Engineer

Remote • San Francisco (CA)

On-site
USD 150,000 - 190,000
Principal Architect - Databricks
Principal Architect - Databricks

shi • United States

On-site
USD 180,000 - 240,000
Diversity & Inclusion
Growth opportunities
Health and wellness benefits
+1
Senior Data Architect – Databricks
Senior Data Architect – Databricks

MatrixEdge Inc • Kansas City (MO)

On-site
USD 150,000 - 190,000
Data Engineer, Principal
Data Engineer, Principal

Horizontal Talent • Golden Valley (MN)

On-site
USD 68,000 - 117,000
Medical, dental, vision benefits
Retirement plan
Principal Databricks Engineer
Principal Databricks Engineer

Seneca Resources • United States

Remote
USD 190,000 - 230,000
Health, dental, and vision coverage
401(k) retirement plans
Dedicated staffing partner
Chief Technology Officer — Databricks Data & AI Platform
Chief Technology Officer — Databricks Data & AI Platform

EPAM Systems • United States

Remote
USD 260,000 - 520,000
Principal Data Platform Architect — Databricks Expert
Principal Data Platform Architect — Databricks Expert

Colibri Digital • United States

Remote
GBP 100,000 - 150,000
Private health insurance
Remote-first work model
Certification access
+1
Databricks Data Architect
Databricks Data Architect

NewVision Software • United States

Remote
USD 120,000 - 180,000