Senior / Principal Data Engineers

Primus Connect; England; On-site; GBP 101,000 - 129,000; Part time

Job summary

Primus Connect is hiring Databricks Data Engineers at Senior and Principal levels for contract engagements in the UK. The role focuses on building production-grade pipelines with Databricks, PySpark, and Spark SQL, designing Delta Lake architectures, and implementing Unity Catalog for governance.

Positions offer remote UK work and flexible start ASAP. Candidates should have strong Databricks expertise, Terraform/IaC, CI/CD, and experience mentoring engineers while collaborating with stakeholders

Qualifications

• Extensive commercial Databricks experience across projects.
• Strong PySpark and Spark SQL skills with Delta Lake knowledge.
• Ability to design scalable data pipelines and governance.

Responsibilities

• Build production-grade data pipelines using Databricks, PySpark and Spark SQL.
• Design Delta Lake and Medallion Architecture solutions.
• Develop scalable ingestion and transformation pipelines.

Skills

Databricks

PySpark

Spark SQL

Medallion Architecture

Unity Catalog

Python

Terraform

CI/CD

Kafka

MLflow

Delta Sharing

Lakehouse Federation

Structured Streaming

Tools

Databricks Asset Bundles

Delta Lake

Job description

£550-£700 per day | Outside IR35 | Fully Remote UK | ASAP

Primus Connect is working with a fast-growing Data & AI Consultancy that is scaling its Databricks engineering capability across multiple client projects.

We’re looking for experienced Databricks Data Engineers at both Senior and Principal level to join the team on an initial contract engagement.

These roles range from hands‑on senior engineering through to principal‑level technical leadership and architecture.

What you’ll be working on

• Building production‑grade data pipelines using Databricks, PySpark and Spark SQL
• Designing Delta Lake and Medallion Architecture solutions
• Developing scalable ingestion and transformation pipelines
• Implementing Unity Catalog for governance, lineage and access control
• Building CI/CD and Infrastructure‑as‑Code using Terraform / Databricks Asset Bundles
• Spark performance optimisation and cost optimisation
• Establishing data engineering standards and best practices
• Supporting and mentoring other engineers
• Working closely with technical and business stakeholders

At Principal level, you’ll also take ownership of platform architecture, set technical direction and act as a technical authority across Databricks engineering workstreams.

What we’re looking for

You’ll need strong commercial experience across:

• Databricks
• PySpark
• Spark SQL
• Medallion Architecture
• Unity Catalog
• Python
• Terraform / Infrastructure-as-Code
• CI/CD

Experience with Databricks Asset Bundles, Delta Sharing, Lakehouse Federation, Structured Streaming, Kafka or MLflow would also be beneficial.

For Principal-level opportunities, we’re particularly interested in people who have previously operated as a Principal Data Engineer, Lead Data Engineer or Databricks Architect and can demonstrate ownership of large-scale Databricks platforms.

Principal Data Engineer: Up to £700 per day

Location: Fully Remote - UK

Start: ASAP

If you’re an experienced Databricks Data Engineer looking for your next contract, apply with your latest CV.