Lead Data Engineer: Hadoop, Spark & Python

Square One Resources; Sheffield; Hybrid; GBP 80,000 - 100,000; Full time

Job summary

A leading recruitment agency is looking for a Lead Data Engineer to join their team in Sheffield. The role involves developing data engineering platforms and integrating large datasets. Ideal candidates should have extensive expertise in Hadoop and Spark, proficiency in Python, and experience working with BI teams. This contract position requires in-office work three days per week, with competitive pay of up to £617 per day until November 2026.

Qualifications

• Proven experience in data engineering platforms.
• Extensive enterprise experience with Hadoop and Spark.
• Proficiency in Python and scripting.

Responsibilities

• Lead development of data engineering platforms.
• Build and optimize ETL/ELT data pipelines.
• Collaborate with BI and Analytics teams.

Skills

Hadoop

Spark

Python

SQL

NoSQL

ETL/ELT

CI/CD

Agile methodologies

SRE culture

Job description

A leading recruitment agency is looking for a Lead Data Engineer to join their team in Sheffield. The role involves developing data engineering platforms and integrating large datasets. Ideal candidates should have extensive expertise in Hadoop and Spark, proficiency in Python, and experience working with BI teams. This contract position requires in-office work three days per week, with competitive pay of up to £617 per day until November 2026.