Databricks Lead Data Engineer - Build & Lead Platform

Big Red Recruitment Midlands Ltd; Kirklees; On-site; GBP 68,000 - 83,000; Full time

Job summary

Big Red Recruitment Midlands Ltd is seeking a Lead Data Engineer to drive the transformation of a multi-site UK organisation to a Databricks-based platform. You will own engineering delivery, stay hands-on, design scalable pipelines and establish data layers, while mentoring a small team of Data Engineers.

The role emphasizes building trusted datasets, improving data quality, and shaping engineering standards across the organisation.

Qualifications

• Strong commercial Data Engineering experience.
• Hands-on Databricks expertise.
• Excellent SQL skills.
• Experience designing and developing production data pipelines.
• Strong understanding of modern data-platform architecture.
• Experience working within complex enterprise data environments.
• Previous team leadership or line-management experience.
• Strong stakeholder-management and communication skills.
• The credibility to lead and develop experienced Data Engineers.

Responsibilities

• Lead Data Engineering delivery across Databricks.
• Remaining hands-on with engineering, development and problem-solving.
• Designing and building scalable, reliable data pipelines.
• Helping establish effective bronze, silver and gold data layers.
• Developing curated, trusted and reusable datasets.
• Improving data quality, structure and provenance.
• Supporting the development of enterprise data models and single sources of truth.
• Leading, mentoring and developing a small team of Data Engineers.
• Prioritising and allocating engineering work across multiple projects.
• Establishing strong engineering standards and ways of working.
• Working closely with Data Architecture, Analytics and wider technology teams.
• Acting as a technical sounding board to the Head of Data Engineering.
• Helping reduce manual data processes through better engineering and automation.
• Building internal Databricks knowledge and capability.
• Deputising for senior Data Engineering leadership when required.

Skills

Data Engineering

Databricks

SQL

Python

Job description

Big Red Recruitment Midlands Ltd is seeking a Lead Data Engineer to drive the transformation of a multi-site UK organisation to a Databricks-based platform. You will own engineering delivery, stay hands-on, design scalable pipelines and establish data layers, while mentoring a small team of Data Engineers.

The role emphasizes building trusted datasets, improving data quality, and shaping engineering standards across the organisation.