Remote Data Engineer, Google Cloud & Real-Time Pipelines

IBM; Boston (MA); Híbrido; USD 215.000 - 253.000; Jornada completa

Descripción de la vacante

IBM Consulting seeks a seasoned Data Engineer specializing in Google Cloud data platforms. You will design, build, and maintain data pipelines across batch and real-time workloads, leveraging DataProc, DataFlow, Pub/Sub, BigQuery and BigTable to support data warehouse and data lake workloads.

You will also implement Open-Source tools like Apache Beam, Apache Airflow, dbt, Spark/Python or Spark/Scala, and manage scheduling via Cloud Scheduler and Cloud Composer, ensuring scalable data solutions

Formación

• Bachelor's degree required.
• Deep expertise in Google Cloud data platforms including DataProc/DataFlow/PubSub/BigQuery/BigTable.
• Proficiency with open-source tools such as Apache Beam, Airflow, dbt, Spark/Python or Spark/Scala.
• Experience building batch and real-time data pipelines for Data Warehouse and Data Lake on Google Cloud.

Responsabilidades

• Design batch and real-time data pipelines for Data Warehouse and Datalake on Google Cloud.
• Develop and maintain data engineering solutions using Google Cloud Storage, BigTable, DataProc, Spark, and Hadoop.
• Manage data platforms with Cloud Scheduler and Cloud Composer (Airflow).
• Optimize data layers for efficient migration and processing on Google Cloud services.
• Ensure scalable, efficient pipelines meeting business needs.

Conocimientos

Google Data Platforms

Open-Source Tech

Batch & Real-time

Data Pipeline Design

Data Platform Mgmt

Educación

Bachelor's Degree

Herramientas

Apache Beam

Apache Airflow

dbt

Spark

BigQuery

Dataflow

Descripción del empleo

IBM Consulting seeks a seasoned Data Engineer specializing in Google Cloud data platforms. You will design, build, and maintain data pipelines across batch and real-time workloads, leveraging DataProc, DataFlow, Pub/Sub, BigQuery and BigTable to support data warehouse and data lake workloads.

You will also implement Open-Source tools like Apache Beam, Apache Airflow, dbt, Spark/Python or Spark/Scala, and manage scheduling via Cloud Scheduler and Cloud Composer, ensuring scalable data solutions