Lead Data Engineer

VIQU IT Recruitment; Leeds; Hybrid; GBP 65,000 - 70,000; Full time

Job summary

A leading supply chain organization is seeking a Lead Data Engineer to manage a team of eight and support ongoing digital transformation efforts. This position focuses on developing the Databricks platform, enhancing data governance, and optimizing costs. Ideal candidates will possess extensive experience in data engineering, notably with Python, SQL, and team leadership skills.

Qualifications

• 6+ years of hands-on data engineering experience.
• 1+ year in a managerial role with coaching experience.
• Deep knowledge of the Databricks platform.

Responsibilities

• Leading and mentoring a team of eight engineers.
• Managing the adoption of automated CI/CD pipelines.
• Developing a new Databricks system in AWS or Azure.

Skills

Data Engineering

Python

SQL

People Management

Tools

Databricks

AWS

Azure

GCP

Job description

Role: Lead Data Engineer

Salary: £65,000- £70,000 per annum

Location: Leeds (One day on site).

VIQU has partnered with a leading supply chain organisation looking to expand their data teams. The Lead Data Engineer will manage a team of eight, supporting an ongoing digital transformation. The ideal candidate will have a technical background with recent managerial experience focused on mentoring, coaching, reviewing code, and setting standards. The role involves developing the client's Databricks platform (AWS preferred, open to Azure/GCP), using Python and SQL, contributing to CI/CD pipelines, strategy development, cost optimisation, and data governance frameworks.

Responsibilities include:

1. Leading and mentoring a team of eight engineers.
2. Managing the adoption of automated CI/CD pipelines.
3. Developing a new Databricks system in AWS (or Azure).
4. Optimising costs and establishing data governance frameworks for secure data handling.

Requirements:

• 6+ years of hands-on data engineering experience.
• At least 1 year in a managerial role coaching similar-sized teams.
• Deep knowledge of the Databricks platform.
• Proven Python development skills.
• SQL optimisation expertise.
• Experience with large-scale data pipeline optimisation.
• Experience with Streaming and Batch Spark workloads.
• Strong people management skills.

This job is active. To discuss this opportunity, please APPLY NOW for a no-obligation chat with your VIQU Consultant. You can also contact Jack McManus directly.

If you know someone suitable for this role, VIQU offers an introduction fee up to £1,000 upon successful placement (terms apply).

Follow us on LinkedIn ‘VIQU IT Recruitment’ and Twitter: @VIQU_UK for updates on other opportunities and news.