Senior Data Engineer - Real-Time Data Pipelines & Analytics

LTM; Sheffield; On-site; GBP 76,000 - 103,000; Full time

Job summary

LTM is seeking a Senior Data Engineer to design, build and maintain large-scale data pipelines and ETL workflows in a regulated enterprise environment.

The role is based in Sheffield, UK, for a 6-month contract with a rate of 65 GBP per hour inside IR35. You will ingest, transform and manage data assets for analytics platforms, collaborating with DevOps and cybersecurity teams while optimizing cloud-based data ingestion infrastructure.

Qualifications

• Experience with SRE principles and Azure DevOps.
• Strong scripting using Bash, PowerShell, Azure CLI.
• Programming experience with Python and Java, OO development.
• Strong SQL and KQL experience.
• GitHub source control and CI/CD pipelines.
• Linux administration and scripting experience.
• Networking, operating systems, storage, and infrastructure fundamentals.
• Experience in Data Acquisition and Cloud-Based Data Pipelines.
• Experience with Data Transport, Data Cleaning and Data Integration.
• Strong data engineering pipeline automation and productionization skills.
• ETL workflow design and large-scale data pipelines.
• Data warehousing, data modeling and data cataloguing knowledge.
• Real-Time Analytics and Streaming Data Platforms experience.
• Cloud Cost Optimization experience, Azure-focused.

Responsibilities

• Design, build and maintain largescale data pipelines and ETL workflows across enterprise datasets.
• Ingest, provision and manage raw, enriched and curated data assets for analytics platforms.
• Improve reliability, scalability and real-time data ingestion capabilities.
• Support and enhance cloud-based data ingestion infrastructure and platform services.
• Acquire, transform and integrate data from internal and external sources.
• Perform exploratory data analysis and onboard new data sources.
• Develop automated data engineering workflows and optimize data processing pipelines.
• Manage data modelling, data warehousing, data integration and data cataloguing activities.
• Support real-time analytics and streaming data platforms.
• Contribute to cloud infrastructure engineering, platform operations and production support.
• Implement cloud cost optimization and platform efficiency improvements.
• Collaborate with engineering, analytics, infrastructure and cybersecurity teams.
• Support DevOps automation, monitoring and operational excellence initiatives.
• Maintain documentation, governance standards and operational procedures.

Skills

SRE principles

Azure DevOps

Bash & PowerShell

Azure CLI

Python / Java

SQL / KQL

GitHub CI/CD

Linux administration

Networking fundamentals

Azure Data Factory

Azure Databricks

Kubernetes

Terraform

Spark Streaming

SIEM SOAR

Job description

LTM is seeking a Senior Data Engineer to design, build and maintain large-scale data pipelines and ETL workflows in a regulated enterprise environment.

The role is based in Sheffield, UK, for a 6-month contract with a rate of 65 GBP per hour inside IR35. You will ingest, transform and manage data assets for analytics platforms, collaborating with DevOps and cybersecurity teams while optimizing cloud-based data ingestion infrastructure.