Cloud Data Engineer - Google Cloud Pipelines

IBM; Boston (MA); A distancia; USD 131.000 - 154.000; Jornada completa

Descripción de la vacante

IBM Consulting is seeking a Data Engineer specializing in Google's data platforms to design, build, and maintain data pipelines on Google's Cloud ecosystem. You will implement batch and real-time data flows, migrate data between systems, and optimize the data layer for efficient storage and retrieval.

The role emphasizes hands-on work with DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, Cloud Scheduler, and Cloud Composer, enabling scalable data solutions for clients across

Formación

• Exposure to designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem.

Responsabilidades

• Design batch and real-time data pipelines for Data Warehouse and Datalake using Google services (DataProc, DataFlow, PubSub, BigQuery, BigTable).
• Develop data engineering solutions using Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and DataFlow with Apache Beam or Python.
• Schedule and manage the data platform with Cloud Scheduler and Cloud Composer (Airflow).
• Develop and implement data migration solutions using Google services.
• Design and optimize the data layer using BigQuery, BigTable, and Cloud Spanner for efficient storage and retrieval.

Conocimientos

Google Cloud ecosystem

Data pipeline development

Google Cloud services

Data platform management

Data layer design

Educación

Bachelor's Degree

Herramientas

Google DataProc

DataFlow

Pub/Sub

BigQuery

BigTable

Cloud Spanner

CloudSQL

AlloyDB

Apache Airflow

dbt

Spark

Hadoop

Descripción del empleo

IBM Consulting is seeking a Data Engineer specializing in Google's data platforms to design, build, and maintain data pipelines on Google's Cloud ecosystem. You will implement batch and real-time data flows, migrate data between systems, and optimize the data layer for efficient storage and retrieval.

The role emphasizes hands-on work with DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, Cloud Scheduler, and Cloud Composer, enabling scalable data solutions for clients across