Senior Data Engineer: Build Scalable Cloud Data Pipelines

JACOBS; Greater Manchester; Hybrid; GBP 60,000 - 90,000; Full time

Job summary

Jacobs invites you to join our Data & Digital team in the UK to design, build and maintain modern data platforms across government, defence and regulated sectors.

You will develop scalable data pipelines using PySpark, SQL, Python and Java, collaborating with engineers, architects and client stakeholders to enable trusted decision-making. This role offers hybrid working and opportunities to grow through mentoring and leadership development.

Qualifications

• Experience with cloud-based data engineering architectures (AWS/Azure) and modern data platforms.
• Hands-on with ETL/ELT data pipelines, data quality, validation and monitoring.

Responsibilities

• Design, build and maintain reliable ETL/ELT data pipelines using AWS or Azure cloud-native services and established engineering patterns
• Develop and support data integration components and services, including Java-based microservices where required
• Prepare, transform and validate structured and semi-structured data, including formats such as parquet, avro and JSON
• Write efficient SQL to query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse
• Build scalable data processing solutions using PySpark, Python and cloud-native orchestration services
• Implement data quality checks, validation processes, monitoring and logging to ensure trusted and reliable data delivery
• Support the performance tuning and optimisation of data pipelines, SQL workloads and integration services
• Work with AWS services such as S3, Glue, Lambda, Athena and Step Functions, or Azure equivalents including Data Lake Storage Gen2, Data Factory, Synapse, Azure Functions and Logic Apps
• Contribute to documentation, runbooks, data dictionaries and reusable engineering assets that support delivery teams
• Collaborate with data architects, analysts, software engineers and client stakeholders to deliver high-quality solutions
• Support DevOps ways of working through source control, automated testing and CI/CD practices
• Communicate progress, risks and technical decisions effectively with both technical and non-technical stakeholders

Skills

AWS Glue

Lambda

Step Functions

Azure Data Factory

Synapse

Azure Functions

Logic Apps

PySpark

SQL

Python

Java

ETL/ELT pipelines

Data quality/monitoring

Git

CI/CD

Agile delivery

Senior lead experience

SC Clearance

Tools

AWS Glue

Lambda

Step Functions

Azure Data Factory

Synapse Pipelines

Azure Functions

Logic Apps

PySpark

SQL

Python

Java

Job description

Jacobs invites you to join our Data & Digital team in the UK to design, build and maintain modern data platforms across government, defence and regulated sectors.

You will develop scalable data pipelines using PySpark, SQL, Python and Java, collaborating with engineers, architects and client stakeholders to enable trusted decision-making. This role offers hybrid working and opportunities to grow through mentoring and leadership development.