Lead Data Engineer — Remote Data Platform Architect

CreateFuture; Leeds; Hybrid; GBP 90,000 - 130,000; Full time

Benefits offered by this job

Hybrid work options

Remote options

Office hubs in UK & Bulgaria

Job summary

CreateFuture in the UK is seeking a Lead Data Engineer to own the technical direction of a data engagement, working with Microsoft Fabric, Python and PySpark across our data team. You will lead a team of data engineers, setting standards for pipeline design, data modelling and operational readiness while remaining hands-on with complex challenges.

You will collaborate with client stakeholders and the wider engineering capability to ensure delivery is technically excellent and commercially sound,

Qualifications

• Experience with modern cloud data platforms such as Microsoft Fabric, Databricks, Snowflake or Synapse and designing production data platforms.
• Strong SQL, data modelling and architecture knowledge.
• Hands-on Python and PySpark for batch data processing and orchestration.

Responsibilities

• Own the technical direction of a data engagement, setting standards for pipeline architecture, data modelling, testing, and operational readiness.
• Lead and technically direct a team of data engineers, allocating work, reviewing designs and code, and unblocking technical challenges.
• Design and evolve data platform architecture on Microsoft Fabric, including data pipelines, Azure OneLake, and the broader Azure ecosystem.
• Set and enforce CI/CD, testing, and operational practices across the engagement's data engineering workflows, using Azure DevOps.
• Build professional relationships with key client stakeholders, translating business and data requirements into a coherent technical roadmap.
• Troubleshoot and resolve the most complex issues in live data pipelines, and lead incident response when things go wrong.

Skills

Data engineering

Python

PySpark

SQL

Azure

CI/CD

Azure DevOps

Databricks

Tools

Microsoft Fabric

Databricks

Snowflake

Synapse

Job description

CreateFuture in the UK is seeking a Lead Data Engineer to own the technical direction of a data engagement, working with Microsoft Fabric, Python and PySpark across our data team. You will lead a team of data engineers, setting standards for pipeline design, data modelling and operational readiness while remaining hands-on with complex challenges.

You will collaborate with client stakeholders and the wider engineering capability to ensure delivery is technically excellent and commercially sound,