GCP Data Engineer: Real-Time Pipelines & BigQuery

Capgemini; New York (NY); On-site; USD 96,000 - 115,000; Full time

Benefits offered by this job

Paid time off

Medical/dental/vision

401(k)

Life and disability insurance

Employee assistance programs

Job summary

Capgemini in the United States (New York) seeks a skilled GCP Data Engineer to design, build, and maintain scalable data pipelines on Google Cloud Platform, handling large datasets and cloud-based data services.

You will develop batch and real-time pipelines with BigQuery, Dataflow, Dataproc, Cloud Storage, and Pub/Sub, optimize performance, and collaborate with analysts, data scientists, and business stakeholders to meet reporting needs. This is a full-time role with competitive benefits.

Qualifications

• 5+ years of experience in Data Engineering.
• Hands-on experience with Google Cloud Platform (GCP).
• Strong knowledge of BigQuery, Dataflow, Dataproc, Cloud Storage, and Pub/Sub.
• Experience with SQL and Python.
• Knowledge of ETL/ELT development and data warehousing concepts.
• Familiarity with Spark and distributed data processing frameworks.
• Experience with workflow orchestration tools such as Airflow/Cloud Composer.
• Understanding of CI/CD and version control tools like Git.

Responsibilities

• Design and develop batch and real-time data pipelines on GCP.
• Build and maintain data integration and ETL/ELT processes.
• Work with structured and unstructured data from multiple sources.
• Develop solutions using GCP services such as BigQuery, Dataflow, Dataproc, Cloud Storage, and Pub/Sub.
• Optimize data pipelines for performance, scalability, and reliability.
• Collaborate with data analysts, data scientists, and business stakeholders to support reporting and analytics requirements.
• Implement data quality, monitoring, and governance best practices.
• Troubleshoot and resolve data-related issues.

Skills

GCP

SQL

Python

ETL/ELT

Spark

Airflow

CI/CD

Git

Tools

BigQuery

Dataflow

Dataproc

Cloud Storage

Pub/Sub

Airflow/Cloud Composer

Git

Job description

Capgemini in the United States (New York) seeks a skilled GCP Data Engineer to design, build, and maintain scalable data pipelines on Google Cloud Platform, handling large datasets and cloud-based data services.

You will develop batch and real-time pipelines with BigQuery, Dataflow, Dataproc, Cloud Storage, and Pub/Sub, optimize performance, and collaborate with analysts, data scientists, and business stakeholders to meet reporting needs. This is a full-time role with competitive benefits.