GCP Data Engineer - BigQuery & Dataflow Pipelines (Hybrid)

Involved Solutions; Greater London; Hybrid; GBP 68,000 - 108,000; Full time

Benefits offered by this job

Hybrid work arrangement

Bonus and benefits

Job summary

Involved Solutions is hiring a Data Engineer for a London-based hybrid role. You will design and implement scalable data pipelines on GCP, leveraging BigQuery, Dataflow and Spanner, and work across ETL/ELT processes in a complex enterprise environment.

The role requires strong coding skills (Python/Java/Scala), exposure to Airflow and CI/CD, and a deep understanding of data governance, security and performance optimization.

Qualifications

• Strong experience as a Data Engineer within GCP environments.
• Hands-on experience with BigQuery, Dataflow and Spanner.
• Strong programming capability in Python, Java or Scala.
• Experience with Apache Beam / Dataflow and distributed processing frameworks.
• Experience designing and managing ETL / ELT pipelines.
• Solid understanding of data modelling and database design.
• Experience with workflow orchestration tools such as Apache Airflow.
• Strong understanding of data governance, security and scalability.
• Experience with Pub/Sub, Cloud Composer or Cloud Data Fusion.
• Exposure to CI/CD, DevOps and Infrastructure as Code.
• Experience with real-time streaming architectures.

Responsibilities

• Design and build scalable data pipelines, ETL/ELT workflows and cloud data architectures on GCP.
• Develop solutions using services such as BigQuery, Dataflow, Spanner and Cloud Storage.
• Build and maintain code using Python, Java or Scala for data transformation and processing.
• Optimise data pipelines, queries and workloads for performance and scalability.
• Implement data quality, validation and governance controls.
• Collaborate with cross-functional teams to deliver end-to-end data solutions.
• Ensure alignment with security, privacy and compliance standards.
• Troubleshoot issues across pipelines and processing workflows.
• Maintain documentation across data flows, platforms and architectures.

Skills

BigQuery

Dataflow

Spanner

Python

Java

Scala

Apache Beam

ETL/ELT

Airflow

GCP

CI/CD

DevOps

Data governance

Pub/Sub

Cloud Composer

Real-time streaming

Tools

CI/CD

Cloud Composer

Pub/Sub

Job description

Involved Solutions is hiring a Data Engineer for a London-based hybrid role. You will design and implement scalable data pipelines on GCP, leveraging BigQuery, Dataflow and Spanner, and work across ETL/ELT processes in a complex enterprise environment.

The role requires strong coding skills (Python/Java/Scala), exposure to Airflow and CI/CD, and a deep understanding of data governance, security and performance optimization.