Cloud Data Engineer (AWS/Azure) — ETL, PySpark & Pipelines

Jacobs; Glasgow; Hybrid; GBP 55,000 - 90,000; Full time

Benefits offered by this job

Pension

Holiday allowance

Flexible working

Mentoring and development

Learning time for certifications

Volunteer and community initiatives

Job summary

Jacobs is hiring for a Data Engineer to design, build and operate cloud-based data pipelines across government and regulated environments in the UK. You will work with AWS and Azure services, PySpark, SQL and Java/Python to deliver robust data platforms, partnering with architects, analysts and stakeholders to ensure trusted data delivery and scalable analytics.

You will contribute to documentation, runbooks and reusable assets while practicing DevOps with CI/CD, automated tests and version

Qualifications

• AWS Glue, Lambda, Step Functions or Azure Data Factory, Synapse Pipelines, Azure Functions and Logic Apps
• PySpark
• SQL (Athena, Redshift, Azure Synapse SQL)
• AWS S3 or Azure Data Lake Storage Gen2
• Python and/or Java
• ETL/ELT data pipeline development
• Data quality, validation and monitoring
• Git, CI/CD and Agile delivery practices
• Eligibility for SC Clearance

Responsibilities

• Design, build and maintain reliable ETL/ELT data pipelines using AWS or Azure cloud-native services and established engineering patterns
• Develop and support data integration components and services, including Java-based microservices where required
• Prepare, transform and validate structured and semi-structured data, including formats such as parquet, avro and JSON
• Write efficient SQL to query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse
• Build scalable data processing solutions using PySpark, Python and cloud-native orchestration services
• Implement data quality checks, validation processes, monitoring and logging to ensure trusted and reliable data delivery
• Support the performance tuning and optimisation of data pipelines, SQL workloads and integration services
• Work with AWS services such as S3, Glue, Lambda, Athena and Step Functions, or Azure equivalents including Data Lake Storage Gen2, Data Factory, Synapse, Azure Functions and Logic Apps
• Contribute to documentation, runbooks, data dictionaries and reusable engineering assets that support delivery teams
• Collaborate with data architects, analysts, software engineers and client stakeholders to deliver high-quality solutions
• Support DevOps ways of working through source control, automated testing and CI/CD practices
• Communicate progress, risks and technical decisions effectively with both technical and non-technical stakeholders

Job description

Jacobs is hiring for a Data Engineer to design, build and operate cloud-based data pipelines across government and regulated environments in the UK. You will work with AWS and Azure services, PySpark, SQL and Java/Python to deliver robust data platforms, partnering with architects, analysts and stakeholders to ensure trusted data delivery and scalable analytics.

You will contribute to documentation, runbooks and reusable assets while practicing DevOps with CI/CD, automated tests and version