Google Cloud Platform Data Engineer

PA Consulting; Bagstone; On-site; GBP 85,000 - 115,000; Full time

Benefits offered by this job

Private medical insurance

25 days annual leave

Company pension scheme

Annual performance-based bonus

Life and Income protection insurance

Cycle to work benefits

Tax efficient benefits

Job summary

PA Consulting in the United Kingdom is seeking a Principal GCP Data Engineer to lead data-driven solutions for clients, leveraging BigQuery, Dataflow and Dataproc. You will drive architecture decisions and ensure robust, scalable data platforms.

You will guide development, foster best practices, mentoring engineers, and collaborating with stakeholders in an Agile environment to deliver high-impact data projects across multiple client engagements.

Qualifications

• Experience delivering production-ready data processing solutions on Google Cloud Platform (GCP).
• Experience with batch and streaming pipelines using Apache Spark and Apache Beam.
• Expert understanding of data storage technologies and data lakes, data warehouses and metadata management.
• Ability to translate business requirements into technical specifications and solution designs.

Responsibilities

• Lead data engineering delivery for client projects on GCP.
• Design and implement end-to-end data pipelines using Dataflow, Dataproc and BigQuery.
• Mentor team members and influence senior stakeholders across projects.
• Establish CI/CD, automated testing and deployments for data solutions.

Skills

BigQuery

Dataflow

Dataproc

Apache Spark

Apache Beam

ETL/ELT design

Event-driven architectures

CI/CD pipelines

Git

Agile/Scrum/Kanban

Kafka

Relational databases

GCP

Tools

Google Cloud Platform

Cloud Composer

Dataproc

BigQuery

Kafka

Relational DBs (PostgreSQL/MySQL/Oracle/SQL Server)

Collibra/Data Catalog

Job description

engineering standards and practices, ensuring we maintain a shared, modern and robust approach.Lead and influence technical discussions with client stakeholders to achieve the collective buy-in required to be successfulCoach and mentor team members, regardless of seniority, and work with them to build their expertise and understanding.QualificationsTo be successful in this role, you will need to have:Experience delivering and deploying production-ready data processing solutions using BigQuery, Pub/Sub, Dataflow and DataprocExperience developing end-to-end solutions using batch and streaming frameworks such as Apache Spark and Apache Beam.Expert understanding of when to use a range of data storage technologies including relational/non-relational, document, row-based/columnar data stores, data warehousing and data lakes.Expert understanding of data pipeline patterns and approaches such as event-driven architectures, ETL/ELT, stream processing and data working with business owners to
translate business requirements into technical specifications and solution designs that satisfies the data requirements of the business.Experience working with metadata management products such as Cloud Data Catalog and Collibra and Data Governance tools like DataplexExperience in developing solutions on GCP using cloud-native principles and building data quality alerting and data quarantine solutions to ensure downstream datasets can be trusted.Experience implementing CI/CD pipelines using techniques including as git code control/branching, automated tests and automated deployments.Comfortable working in an Agile team using Scrum or Kanban methodologies.In addition to the above, we would be thrilled if you also had:Experience of working on migrations of enterprise scale data platforms including Hadoop and traditional data warehousesAn understanding of machine learning model development lifecycle, feature engineering, training and testingGood understanding or hands-on
experience of KafkaExperience as a DBA or developer on RDBMS such as PostgreSQL, MySQL, Oracle or SQL ServerExperience designing data applications to meet non-functional requirements such as performance and availabilityPersonal qualitiesYou are pragmatic and already understand that writing code is only part of what a data engineer does.You can clearly communicate with both clients and peers, describing technical issues and solutions in both written and meeting/workshop contexts.You are able to clearly explain technical concepts to non technical audiences at all levels of an are able to influence and persuade senior and specialist client stakeholders, potentially across multiple organisational boundaries without direct authority.You are a confident problem solver and troubleshooter.You are confident and generous in sharing your specialist knowledge, ideas and solutions.You are constantly learning and able to make others better by consciously teaching and
unconsciously inspiring.Additional InformationBenefits package at PA:Private medical insuranceInterest free season ticket loan25 days annual leave with the opportunity to buy 5 additional daysCompany pension schemeAnnual performance-based bonusLife and Income protection insuranceTax efficient benefits (cycle to work, give as you earn, childcare benefits)Voluntary benefits (Dental, critical illness, spouse/partner life assurance)PA is committed to building an inclusive and supportive culture where diversity thrives, and all of our people can excel. We believe that greater diversity stimulates innovation, enabling us to fulfil our purpose of 'Bringing Ingenuity to Life', supporting the growth of our people, and delivering more enduring results for our clients. #LI-NF1SummaryType: Full-timeFunction: ConsultingExperience level: Mid-Senior LevelIndustry: Management Consulting
Company DescriptionBringing Ingenuity to LifeWe're an innovation and transformation consultancy that believes in the power of ingenuity to build a positive-human future in a technology-driven world. Our diverse teams of experts combine innovative thinking with breakthrough-technologies to progress further, faster.With a global network of FTSE 100 and Fortune 500 clients, we'll offer you unrivalled opportunities for growth and the freedom to excel. Combining strategies, technologies and innovation, we turn complexity to opportunity and deliver enduring results, enabling you to build a lasting career.Isn't it time you joined us?Job DescriptionAs a Principal GCP Data Engineer, you'll be a true subject matter expert in using the data processing and management capabilities of Google Cloud to develop data-driven solutions for our clients.You will typically lead a team or the solution delivery effort, demonstrating technical excellence through leading by example. You could be providing
technical support, leading an engineering team or working across multiple teams as a subject matter expert who is critical to the success of a large programme of work.Your team members will look to you as a trusted expert and will expect you to define the end-to-end software development lifecycle in line with modern best practices. As part of your responsibilities, you will be expected to:Develop robust data processing jobs using tools such as Google Cloud Dataflow, Dataproc and BigQueryDesign and deliver automated data pipelines that use orchestration tools such as Cloud ComposerDesign end-to-end solutions and contribute to architecture discussions beyond data processingOwn the development process for your team, building strong principles and putting robust methods and patterns in place across architecture, scope, code quality and deployments.Shape team behaviour for writing specifications and acceptance criteria, estimating stories, sprint planning and define and evolve PA's data