Senior GCP Data Engineer - BigQuery, Dataflow & Spark

PA Consulting; Bagstone; On-site; GBP 85,000 - 115,000; Full time

Benefits offered by this job

Private medical insurance

25 days annual leave

Company pension scheme

Annual performance-based bonus

Life and Income protection insurance

Cycle to work benefits

Tax efficient benefits

Job summary

PA Consulting in the United Kingdom is seeking a Principal GCP Data Engineer to lead data-driven solutions for clients, leveraging BigQuery, Dataflow and Dataproc. You will drive architecture decisions and ensure robust, scalable data platforms.

You will guide development, foster best practices, mentoring engineers, and collaborating with stakeholders in an Agile environment to deliver high-impact data projects across multiple client engagements.

Qualifications

• Experience delivering production-ready data processing solutions on Google Cloud Platform (GCP).
• Experience with batch and streaming pipelines using Apache Spark and Apache Beam.
• Expert understanding of data storage technologies and data lakes, data warehouses and metadata management.
• Ability to translate business requirements into technical specifications and solution designs.

Responsibilities

• Lead data engineering delivery for client projects on GCP.
• Design and implement end-to-end data pipelines using Dataflow, Dataproc and BigQuery.
• Mentor team members and influence senior stakeholders across projects.
• Establish CI/CD, automated testing and deployments for data solutions.

Skills

BigQuery

Dataflow

Dataproc

Apache Spark

Apache Beam

ETL/ELT design

Event-driven architectures

CI/CD pipelines

Git

Agile/Scrum/Kanban

Kafka

Relational databases

GCP

Tools

Google Cloud Platform

Cloud Composer

Dataproc

BigQuery

Kafka

Relational DBs (PostgreSQL/MySQL/Oracle/SQL Server)

Collibra/Data Catalog

Job description

PA Consulting in the United Kingdom is seeking a Principal GCP Data Engineer to lead data-driven solutions for clients, leveraging BigQuery, Dataflow and Dataproc. You will drive architecture decisions and ensure robust, scalable data platforms.

You will guide development, foster best practices, mentoring engineers, and collaborating with stakeholders in an Agile environment to deliver high-impact data projects across multiple client engagements.