GCP Data Engineer

Capgemini; New York (NY); On-site; USD 120,000 - 180,000; Full time

Job summary

Capgemini in New York seeks a highly skilled GCP Data Engineer to design, build, and optimize scalable data pipelines and cloud-based data solutions.

You will work with BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Composer, Looker Studio and Dataform to support BI, analytics, and AI/ML initiatives, collaborating with Data Analysts and Scientists.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related discipline.
• 4+ years of Data Engineering experience.
• 2+ years of hands-on GCP experience.
• Experience working in Agile/Scrum environments.

Responsibilities

• Design, develop, and maintain scalable data pipelines on Google Cloud Platform.
• Build and optimize ETL/ELT processes for ingesting data from multiple sources.
• Develop data solutions using BigQuery, Dataflow, Dataproc, Cloud Storage, Pub/Sub, and Cloud Composer.
• Create and manage enterprise-scale data warehouses and data lakes.
• Implement batch and real-time data processing frameworks.
• Ensure data quality, governance, security, and compliance standards.
• Optimize BigQuery queries and cloud infrastructure for performance and cost efficiency.
• Work closely with Data Analysts, Data Scientists, Business Analysts, and Application Teams.
• Monitor pipeline performance and troubleshoot data-related issues.
• Support CI/CD implementation and automation of data engineering workflows.
• Document technical designs, architecture, and operational procedures.
• Participate in cloud migration and modernization initiatives.

Skills

GCP

BigQuery

Dataflow

Dataproc

Pub/Sub

Cloud Composer

Looker Studio

ETL/ELT

Programming: Python/Java/Scala

Data Modeling

Spark/Hadoop

CI/CD

Docker

Terraform

Git

Airflow

Kafka

Education

Bachelor's or Master's in CS/IT/Engineering

Tools

Dataform

Cloud Functions

Job description

We are seeking a highly skilled Google Cloud Platform (GCP) Data Engineer to design, build, and optimize scalable data pipelines and cloud-based data solutions. The ideal candidate will have hands-on experience with GCP services, modern data architectures, ETL/ELT processes, and data warehousing technologies to support business intelligence, analytics, and AI/ML initiatives.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines on Google Cloud Platform.
• Build and optimize ETL/ELT processes for ingesting data from multiple sources.
• Develop data solutions using BigQuery, Dataflow, Dataproc, Cloud Storage, Pub/Sub, and Cloud Composer.
• Create and manage enterprise-scale data warehouses and data lakes.
• Implement batch and real-time data processing frameworks.
• Ensure data quality, governance, security, and compliance standards.
• Optimize BigQuery queries and cloud infrastructure for performance and cost efficiency.
• Work closely with Data Analysts, Data Scientists, Business Analysts, and Application Teams.
• Monitor pipeline performance and troubleshoot data-related issues.
• Support CI/CD implementation and automation of data engineering workflows.
• Document technical designs, architecture, and operational procedures.
• Participate in cloud migration and modernization initiatives.

Required Skills

Technical Skills

• Strong experience with Google Cloud Platform (GCP).
• Hands-on expertise with:
• BigQuery
• Dataflow
• Pub/Sub
• Cloud Composer (Airflow)
• Cloud Functions
• Experience with Python, Java, or Scala.
• Expertise in ETL/ELT development and data integration.
• Knowledge of data modeling techniques (Star Schema, Snowflake Schema).
• Experience with Apache Spark and Hadoop ecosystem.
• Understanding of data warehousing concepts.
• Familiarity with version control tools such as Git.
• Exposure to CI/CD tools and DevOps practices.
• BigQuery
• Looker / Looker Studio
• Dataform
• Terraform
• Docker
• Kafka (preferred)

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related discipline.
• 4+ years of Data Engineering experience.
• 2+ years of hands-on GCP experience.
• Experience working in Agile/Scrum environments.
• Excellent problem-solving and communication skills.

Preferred Qualifications

• GCP Professional Data Engineer Certification.
• Experience supporting AI/ML workloads.
• Knowledge of Data Governance and Data Quality frameworks.
• Financial Services, Banking, Insurance, or Healthcare domain experience preferred.
• Experience migrating on-premise data platforms to GCP.

Nice to Have

• Vertex AI knowledge.
• Real-time streaming using Pub/Sub and Dataflow.
• Experience with Informatica, Talend, or other ETL tools.