Data Engineer

Square One Resources; Sheffield; Hybrid; GBP 55,000 - 78,000; Part time

Job summary

Square One Resources is recruiting a Senior Data Engineer to design, build and maintain large-scale data pipelines and cloud platforms in a financial services context. The role emphasizes real-time analytics, data warehousing and robust data governance, with hybrid working in the UK.

The candidate will work across Azure-based tooling (ADF, Databricks, Synapse), Python/SQL, and CI/CD pipelines, collaborating with analytics, infrastructure and cybersecurity teams to optimise costs and secure

Qualifications

• Design, build and maintain large-scale data pipelines and ETL workflows across enterprise datasets.
• Ingest, provision, and manage raw, enriched, and curated data assets for analytics platforms
• Improve reliability, scalability and real-time data ingestion capabilities
• Support and enhance cloud-based data ingestion infrastructure and platform services
• Acquire, transport, clean, transform and integrate data from internal and external sources
• Perform exploratory data analysis and onboard new data sources
• Develop automated data engineering workflows and optimise data processing pipelines
• Manage data modelling, data warehousing, data integration and data cataloguing activities
• Support real-time analytics and streaming data platforms
• Contribute to cloud infrastructure engineering, platform operations and production support
• Implement cloud cost optimisation and platform efficiency improvements
• Collaborate with engineering, analytics, infrastructure and cybersecurity teams
• Support DevOps automation, monitoring and operational excellence initiatives
• Maintain documentation, governance standards and operational procedures

Responsibilities

• Design, build and maintain large-scale data pipelines and ETL workflows across enterprise datasets
• Ingest, provision, and manage raw, enriched, and curated data assets for analytics platforms
• Improve reliability, scalability and real-time data ingestion capabilities
• Support and enhance cloud-based data ingestion infrastructure and platform services
• Acquire, transport, clean, transform and integrate data from internal and external sources
• Perform exploratory data analysis and onboard new data sources
• Develop automated data engineering workflows and optimise data processing pipelines
• Manage data modelling, data warehousing, data integration and data cataloguing activities
• Support real-time analytics and streaming data platforms
• Contribute to cloud infrastructure engineering, platform operations and production support
• Implement cloud cost optimisation and platform efficiency improvements
• Collaborate with engineering, analytics, infrastructure and cybersecurity teams
• Support DevOps automation, monitoring and operational excellence initiatives
• Maintain documentation, governance standards and operational procedures

Skills

Azure

Azure DevOps

Python

SQL

KQL

Bash

PowerShell

Azure CLI

Linux

GitHub

CI/CD

Networking

Kubernetes

Terraform

Spark Streaming

Tools

Azure Data Factory

Azure Databricks

Unity Catalog

Azure Data Lake

Synapse Analytics

Azure Functions

Azure Logic Apps

Azure Monitor

Power BI

Kafka

Prometheus

Grafana

Elasticsearch

Job description

Data Engineer

Location: Sheffield, UK
Contract: 6 Months
Rate: £300 - 425 per day, Inside IR35
Working Pattern: Some hybrid working
Start Date: September 2026

Role Overview

We are currently recruiting for a Senior Data Engineer to support a leading financial services customer. The role will focus on designing, building and maintaining large-scale data pipelines, cloud-based data platforms, and data engineering workflows.

Responsibilities

• Design, build and maintain large-scale data pipelines and ETL workflows across enterprise datasets

• Ingest, provision, and manage raw, enriched, and curated data assets for analytics platforms

• Improve reliability, scalability and real-time data ingestion capabilities

• Support and enhance cloud-based data ingestion infrastructure and platform services

• Acquire, transport, clean, transform and integrate data from internal and external sources

• Perform exploratory data analysis and onboard new data sources

• Develop automated data engineering workflows and optimise data processing pipelines

• Manage data modelling, data warehousing, data integration and data cataloguing activities

• Support real-time analytics and streaming data platforms

• Contribute to cloud infrastructure engineering, platform operations and production support

• Implement cloud cost optimisation and platform efficiency improvements

• Collaborate with engineering, analytics, infrastructure and cybersecurity teams

• Support DevOps automation, monitoring and operational excellence initiatives

• Maintain documentation, governance standards and operational procedures

Technical & Engineering Skills

• Strong experience with Azure and Azure DevOps
• Strong Python and SQL skills, including KQL
• Experience with Bash, PowerShell and Azure CLI
• Strong Linux administration and scripting
• Experience with GitHub, source control and CI/CD pipelines
• Understanding of networking, operating systems, storage and infrastructure

Data Engineering

• Experience designing and maintaining large-scale data pipelines and ETL workflows
• Strong experience in data acquisition, transport, cleaning and integration
• Experience with data modelling, data warehousing and data cataloguing
• Experience with pipeline automation, productionisation and optimisation
• Experience supporting real-time analytics and streaming data platforms

Azure Data Platform

• Azure Data Factory
• Azure Databricks and Unity Catalog
• Azure Data Lake and Synapse Analytics
• Azure Functions and Azure Logic Apps
• Azure Monitor and Azure Log Analytics
• Power BI

Cloud & Infrastructure

• Strong knowledge of Azure IAM, networking, compute, storage, databases, and containers
• Experience with Kubernetes and cloud-native platforms
• Experience with Infrastructure as Code, including Terraform, ARM, Chef or Ansible
• Experience with cloud cost optimisation
• Understanding of SRE principles

Streaming & Platform Technologies

• Experience with Azure Event Hubs, Kafka and Spark Streaming
• Knowledge of Nginx, Apache, Cosmos DB, Prometheus, Grafana and Elasticsearch

Disclaimer
Notwithstanding any guidelines given to level of experience sought, we will consider candidates from outside this range if they can demonstrate the necessary competencies.
Square One is acting as both an employment agency and an employment business, and is an equal opportunities recruitment business. Square One embraces diversity and will treat everyone equally. Please see our website for our full diversity statement.