Lead Data Engineer

Big Red Recruitment Midlands Limited; England; Hybrid; GBP 60,000 - 80,000; Full time

Job summary

A premier recruitment company based in the UK is looking for a skilled Data Engineer to join their national Data & Analytics team. This role focuses on the design and implementation of a data warehousing platform using Azure Databricks. Responsibilities include leading data modeling and ETL pipeline development, tuning data platforms, and upskilling team members. The position offers a salary range of £60,000 to £80,000 and a hybrid work model based in Atherstone. Visa sponsorship is not provided.

Qualifications

• Expertise in Azure Databricks is essential for the role.
• Experience in data engineering and architecture is crucial.
• Get involved in data integration processes and large-scale data processing.

Responsibilities

• Lead the design and implementation of a new Databricks-based data warehousing solution.
• Design and develop data models, ETL pipelines, and data integration processes.
• Monitor, tune, and optimise data platforms for reliability and performance.

Skills

Azure Databricks

PySpark

Data modeling

ETL pipeline development

Performance tuning

Job description

Are you a skilled Data Engineer looking to step into the world of architecture?

An exciting opportunity has opened for an experienced Data Engineer to join our national Data & Analytics function at a time of significant technical modernisation.

The team is about to embark on a greenfield project to build a futureproof data warehousing platform using Azure Data Factory and Databricks, with a clear focus on scalability, quality, and best practice.

Working within a specialist Data Platform & Engineering team, you will join as the expert within Azure Databricks and PySpark. You'll help to upskill our teams knowledge of Databricks, get involved in data modelling and ETL pipeline development, integrations, and performance tuning. This is an ever evolving project as the business becomes more and more data driven and your knowledge of Databricks will be pivotal in shaping the architecture.

This is an ideal role for a Lead Data Engineer ready to step into an architectural role, or an established architect keen to take ownership of a highly visible and strategic data platform project.

What you’ll be doing:

• Leading the design and implementation of a new Databricks-based data warehousing solution
• Designing and developing data models, ETL pipelines, and data integration processes
• Large scale data processing using PySpark
• Monitoring, tuning, and optimising data platforms for reliability and performance
• Upskilling the wider team in Databricks best practices, including modern architecture patterns

Location: Atherstone (Hybrid 3 days office / 2 days remote)

Salary: £60 000 to £80 000 depending on experience

If you're passionate about shaping modern data platforms and have the technical expertise to make a measurable impact, we’d love to hear from you.

Apply now as we have interview slots available!

Please note: Visa sponsorship is not provided for this role and we are only considering applications of candidates who have permanent residency in the UK.

We are an equal opportunity recruitment company. This means we welcome applications from all suitably qualified people regardless of race, sex, disability, religion, sexual orientation or age.

We are particularly invested in Neurodiversity inclusion and offer reasonable adjustments in the interview process. Reasonable adjustments are changes that we can make in the interview process if your disability puts you at a disadvantage compared with others who are not disabled. If you would benefit from a reasonable adjustment in your interview process, please call or email one of our recruiters.