Lead Data Engineer - Databricks Platform Builder

Big Red Recruitment; Huddersfield; On-site; GBP 45,000 - 75,000; Full time

Job summary

Big Red Recruitment is seeking a Lead Data Engineer to own the Databricks platform and drive a major data transformation across a multi-site UK organisation. You will lead a small engineering team while remaining actively hands-on with design, pipelines and data products.

The role requires strong Data Engineering experience, Databricks expertise and excellent SQL skills. You will shape data models, improve data quality and establish robust data layers (bronze/silver/gold) across the estate.

Qualifications

• Strong commercial Data Engineering experience.
• Hands-on Databricks expertise.
• Excellent SQL skills.
• Experience designing and developing production data pipelines.
• Strong understanding of modern data-platform architecture.
• Experience working within complex enterprise data environments.
• Previous team leadership or line-management experience.
• The credibility to lead and develop experienced Data Engineers.
• Strong stakeholder-management and communication skills.
• The confidence to challenge, prioritise and make decisions.
• A proactive, pragmatic and delivery-focused approach.
• Python would be useful but isn't essential.

Responsibilities

• Leading Data Engineering delivery across Databricks.
• Remaining hands-on with engineering, development and problem-solving.
• Designing and building scalable, reliable data pipelines.
• Helping establish effective bronze, silver and gold data layers.
• Developing curated, trusted and reusable datasets.
• Improving data quality, structure and provenance.
• Supporting the development of enterprise data models and single sources of truth.
• Leading, mentoring and developing a small team of Data Engineers.
• Prioritising and allocating engineering work across multiple projects.
• Establishing strong engineering standards and ways of working.
• Working closely with Data Architecture, Analytics and wider technology teams.
• Acting as a technical sounding board to the Head of Data Engineering.
• Helping reduce manual data processes through better engineering and automation.
• Building internal Databricks knowledge and capability.
• Deputising for senior Data Engineering leadership when required.

Skills

Data Engineering

Databricks

SQL

Leadership

Tools

Databricks

Job description

Big Red Recruitment is seeking a Lead Data Engineer to own the Databricks platform and drive a major data transformation across a multi-site UK organisation. You will lead a small engineering team while remaining actively hands-on with design, pipelines and data products.

The role requires strong Data Engineering experience, Databricks expertise and excellent SQL skills. You will shape data models, improve data quality and establish robust data layers (bronze/silver/gold) across the estate.