Advanced Cloud Data Engineer - GCP & DataOps

Your World Recruitment Group; Cardiff; On-site; GBP 36,000 - 39,000; Full time

Job summary

Public Health Wales is seeking an Advanced Data Engineer to drive the modernization of national disease surveillance data platforms, migrating from on-premises to Google Cloud Platform to support CDSC. You will design, build, and optimize scalable ETL/ELT pipelines, apply DataOps, and ensure compliance with UK GDPR and NHS data governance while collaborating with stakeholders in an Agile environment.

You'll work with Spark, BigQuery, Airflow, and Docker in a cloud-first setup within Public

Qualifications

• Strong proficiency in SQL and Python; R desirable.
• Experience with cloud-native solutions (GCP), migrating on-prem to the cloud.
• Experience with big data processing (Spark, Dataproc, BigQuery) and orchestration (Airflow, Cloud Composer, Prefect).
• DevOps with Git, Docker, CI/CD and DataOps practices.
• Undergraduate degree in CS or related field.
• GCP Data Engineer or GCP Cloud Architect certification desirable.

Responsibilities

• Data Engineering & Cloud Architecture: Design, build, test, and maintain scalable ETL/ELT pipelines; migrate legacy data platforms to GCP.
• Pipeline Orchestration & Optimization: Monitor, troubleshoot, and optimize pipelines using Spark, BigQuery, Airflow, Cloud Composer, Prefect.
• Software & DataOps Best Practices: Apply CI/CD, containerization (Docker), Git-based version control, automated testing; maintain documentation.
• Data Modeling & Governance: Develop data models; implement metadata, data lineage, validation frameworks; ensure GDPR compliance.
• Collaboration & Delivery: Work in an Agile environment; translate business needs into robust technical solutions; communicate clearly with stakeholders.

Skills

SQL

Python

R (desirable)

Education

Bachelor's degree in Computer Science or related

Tools

Docker

Airflow

Cloud Composer

Prefect

BigQuery

Dataproc

Spark

Git

CI/CD

Job description

Public Health Wales is seeking an Advanced Data Engineer to drive the modernization of national disease surveillance data platforms, migrating from on-premises to Google Cloud Platform to support CDSC. You will design, build, and optimize scalable ETL/ELT pipelines, apply DataOps, and ensure compliance with UK GDPR and NHS data governance while collaborating with stakeholders in an Agile environment.

You'll work with Spark, BigQuery, Airflow, and Docker in a cloud-first setup within Public