Senior Data Engineer (Azure, Spark SQL, Team Lead)

HOK Consulting - Technical Recruitment Consultancy; Sheffield; Hybrid; GBP 55,000 - 65,000; Full time

Job summary

A leading technical recruitment consultancy is seeking a Senior Data Engineer to work on exciting hybrid contract opportunities in the Banking and Financial Services sector. This role involves leading projects focused on data migration to Spark SQL on Azure, utilizing strong SQL skills and team leadership experience in an Agile setting. Candidates should be proficient in ETL workflows and familiar with Azure Databricks.

Qualifications

• Hands-on expertise in SQL/BigQuery migration.
• Experience with Azure and cloud data tools.
• Leadership in Agile/SAFe environments.

Responsibilities

• Lead data migration projects and agile team.
• Migrate BigQuery SQL to Spark SQL.
• Build ETL workflows utilizing Azure Databricks.

Skills

SQL

BigQuery SQL

Spark SQL

Python

ETL pipeline development

Agile

Team leadership

Education

Relevant degree in Computer Science or related field

Tools

Azure Databricks

Job description

Direct message the job poster from HOK Consulting - Technical Recruitment Consultancy

I introduce talented IT Professionals to exciting contract opportunities within Banking & Financial Services

Location: Hybrid (3 days/week from Sheffield)

We are looking for a Senior Data Engineer with hands-on expertise in SQL/BigQuery migration, Azure Databricks, and Spark SQL, who also brings team leadership experience and thrives in Agile/SAFe/Scrum environments.

Key Responsibilities:

• Lead and contribute to a small Agile team working on a cross-cloud data migration project.
• Migrate complex BigQuery SQL transformations to Spark SQL on Azure.
• Build & execute ETL workflows using Azure Databricks and Python.
• Drive automation of SQL workflows and artefact migration across cloud providers.
• Collaborate with developers, POs, and stakeholders on quality delivery and performance optimization.

Key Requirements:

• Strong SQL skills (BigQuery SQL & Spark SQL), Python, and ETL pipeline development.
• Experience with Azure and cloud data tools.
• Familiarity with CI/CD practices and automation.
• Proven team leadership and organisational capabilities.
• Comfortable with Agile/SAFe/PI Planning methodologies.

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Contract

Job function

• Job function

  Information Technology

• Industries

  Information Services and Technology, Information and Media

Referrals increase your chances of interviewing at HOK Consulting - Technical Recruitment Consultancy by 2x

Get notified about new Data Engineer jobs in Greater Sheffield Area.

Sheffield, England, United Kingdom 2 weeks ago

Sheffield, England, United Kingdom 1 month ago

Rotherham, England, United Kingdom £55,000.00-£65,000.00 1 week ago

Sheffield, England, United Kingdom 4 weeks ago

Sheffield, England, United Kingdom 3 days ago

South Yorkshire, England, United Kingdom 14 hours ago

Data Engineering and Reporting Specialist

Barnsley, England, United Kingdom 3 weeks ago

Sheffield, England, United Kingdom 19 hours ago

Doncaster, England, United Kingdom 1 month ago

Sheffield, England, United Kingdom 2 weeks ago

Sheffield, England, United Kingdom 1 week ago

Sheffield, England, United Kingdom 2 days ago

Sheffield, England, United Kingdom 1 week ago

South Yorkshire, England, United Kingdom 1 week ago

South Yorkshire, England, United Kingdom 1 week ago

South Yorkshire, England, United Kingdom 2 weeks ago

Doncaster, England, United Kingdom 5 days ago

Sheffield, England, United Kingdom 3 days ago

South Yorkshire, England, United Kingdom 14 hours ago

Sheffield, England, United Kingdom 2 weeks ago

Rotherham, England, United Kingdom £50,000.00-£60,000.00 2 days ago

Sheffield, England, United Kingdom 1 week ago

Rotherham, England, United Kingdom £50,000.00-£60,000.00 1 week ago

Sheffield, England, United Kingdom 2 days ago

Sheffield, England, United Kingdom 4 weeks ago

South Yorkshire, England, United Kingdom 2 days ago

South Yorkshire, England, United Kingdom 4 days ago

Sheffield, England, United Kingdom 1 week ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.