GCP Data Engineer: Real-Time Pipelines & ML Ready

Capgemini; New York (NY); On-site; USD 75,000 - 96,000; Full time

Job summary

Capgemini in the United States is seeking a highly skilled GCP Data Engineer with strong Python expertise to design, build, and optimize scalable data solutions on Google Cloud Platform. You will develop batch and real-time data pipelines and enable analytics and AI/ML use cases.

Collaborate with analytics, AI/ML, and business teams; implement CI/CD and Cloud Composer; optimize data lakes, BigQuery models, and data processing for performance and cost efficiency.

Qualifications

• 5+ years of overall data engineering or software engineering experience.
• 2+ years of hands-on Google Cloud Platform experience.
• 2+ years of Python development.
• 2+ years building data pipelines (batch and streaming).

Responsibilities

• Develop and optimize ETL/ELT workflows for structured and unstructured data processing using GCP services such as Dataflow, Dataproc, and Pub/Sub.
• Implement event-driven data processing using Cloud Functions and Pub/Sub.
• Build and manage data ingestion frameworks for streaming and batch data sources.
• Design and optimize data lakes and data warehouses using BigQuery and Cloud Storage.
• Develop efficient data models to support analytics, reporting, and machine learning workloads.
• Optimize performance and cost of data pipelines and queries.
• Develop solutions using Python.
• Automate workflows and orchestration using Cloud Composer (Airflow).
• Implement CI/CD pipelines and deployment automation.
• Collaborate with analytics, AI/ML, and business teams for data consumption needs.
• Troubleshoot data issues and perform root cause analysis.
• Continuously improve pipeline reliability, scalability, and performance.

Skills

GCP Data Engineer

Python

Data pipelines

ETL/ELT

Cloud Native

Tools

Dataflow

Dataproc

Pub/Sub

BigQuery

Cloud Composer

Job description

Capgemini in the United States is seeking a highly skilled GCP Data Engineer with strong Python expertise to design, build, and optimize scalable data solutions on Google Cloud Platform. You will develop batch and real-time data pipelines and enable analytics and AI/ML use cases.

Collaborate with analytics, AI/ML, and business teams; implement CI/CD and Cloud Composer; optimize data lakes, BigQuery models, and data processing for performance and cost efficiency.