Senior Data Engineer

Clearwater People Solutions; Guildford; Hybrid; GBP 70,000 - 110,000; Full time

Job summary

Clearwater People Solutions is seeking a Senior Data Engineer to design, develop, and maintain both legacy and modern cloud-based data platforms and pipelines. The role enables efficient data collection, processing, and analysis and is hybrid, with 2-3 days onsite in Guildford.

You will build robust ETL/ELT pipelines, contribute to data architectures, and monitor performance using Azure Data Factory, Synapse, Spark, Python, SQL, and related tools, supporting analytics and AI initiatives.

Qualifications

• Experience designing scalable ETL/ELT pipelines across on-premise and cloud platforms.
• Proven ability to build robust data architectures and warehouses.
• Strong focus on security, compliance, and reliable data delivery.

Responsibilities

• Design, implement, and maintain ETL/ELT pipelines ingesting diverse data sources.
• Contribute to data architectures, warehouses, and data lake solutions.
• Monitor performance, troubleshoot issues, and optimize pipelines.
• Utilise Azure Data Factory, Synapse, Fabric, Spark, Python to enable analytics and AI initiatives.
• Maintain legacy data pipelines until replacement by modern platforms.

Skills

ETL/ELT pipelines

Cloud data platforms

Azure Data Factory

Synapse

Fabric Data Factory

Spark

Python

SQL

T-SQL/PLSQL

SSIS

Oracle

SQL Server

SAP Data Services

Git

DevOps

Tools

Oracle DBMS/SQL Server

Python

SQL

T-SQL/PLSQL

SSIS

SAP Data Services

Azure Data Factory

Fabric Data Factory

Synapse

Fabric

Spark

Notebooks

Git

Job description

Our client is currently recruiting for a Senior Data Engineer to join their team. The Senior Data Engineer will be responsible for designing, developing, and maintaining both legacy and modern cloud-based data platforms and pipelines that enable efficient data collection, processing, and analysis.

This is a hybrid role (2-3 days onsite in Guildford).

Key Responsibilities for the Senior Data Engineer

• Design, implement, and maintain robust ETL/ELT pipelines to ingest and transform data from diverse internal and external sources. Ensure pipelines are scalable, performant, and reliable, including monitoring overnight loads, resolving failures promptly, and implementing automated alerts to maintain timely data availability
• Contribute to the design, build, and optimisation of data architectures, models, warehouses, and data lake solutions that support reporting, analytics, and AI initiatives. Ensure solutions meet performance, scalability, and reliability standards across legacy and modern cloud platforms.
• Monitor and tune system, pipeline, and query performance. Continuously refine pipelines to enhance efficiency, reduce processing times, and drive improvements in scalability, maintainability, and cost-effectiveness.
• Leverage and maintain modern data engineering tools and frameworks (e.g., Azure Data Factory, Synapse, Microsoft Fabric, Spark, Python). Evaluate and recommend emerging technologies that align with organisational strategy and enhance data capabilities.
• Maintain legacy data warehouses and pipelines (e.g. Oracle, SQL Server, SSIS, SAP Data Services) until replacement by a modern data platform and tooling.

Key Experience for the Senior Data Engineer

• Significant experience in data engineering, preferably working across both traditional and cloud-based platforms.
• Proven track record in designing, implementing, and maintaining scalable data architectures and pipelines.
• Experience delivering solutions in complex, data-driven organisations where security, compliance, and reliability are critical.

• Proficiency in some or all relevant technologies and languages (such as Oracle DBMS/SQL Server, Python, SQL, T-SQL/PLSQL, SSIS, SAP Data Services), cloud data platforms (such as Azure, AWS, Snowflake), and data tools (such as Azure Data Factory, Fabric Data Factory, Synapse, Fabric, Spark, Notebooks), source control (Git) and the DevOps methodology.
• Ability to design, maintain, and optimise scalable ETL/ELT pipelines.
• Ability to troubleshoot complex data issues, monitor performance, and implement automated solutions.
• Strong analytical and problem-solving capabilities with an emphasis on performance tuning and scalability.