Google Cloud Data Engineer — Remote (US)

IBM; San Francisco (CA); Remote; USD 120,000 - 180,000; Full time

Job summary

IBM Consulting seeks an experienced Data Engineer to design, build, and maintain data pipelines on Google's Cloud ecosystem. You will leverage DataProc, DataFlow, Pub/Sub, BigQuery and BigTable to deliver scalable data solutions.

You will work with Apache Beam, Airflow, dbt, Spark/Python, and related tech to ensure robust batch and real-time processing, hosted from anywhere in the United States. This role emphasizes data accessibility, security, and performance.

Qualifications

• Deep expertise designing, building and maintaining data engineering on Google Cloud.
• Proficient with open-source tools like Apache Beam, Airflow, dbt and Spark.
• Experience with batch and real-time data pipelines for Data Warehouse/Data Lake.
• Ability to schedule and manage data platforms with Cloud Scheduler and Airflow.
• Design and optimize data layers for efficient migration and processing.

Responsibilities

• Design batch and real-time data pipelines for Data Warehouse and Datalake on Google Cloud.
• Develop and maintain data engineering solutions using Google Cloud Storage, BigQuery, DataProc, DataFlow, and Spark/Python.
• Manage data platforms with Cloud Scheduler and Cloud Composer (Airflow).
• Optimize data layers for efficient data migration and processing.
• Ensure pipelines scale to meet business needs.

Skills

Google Cloud data platforms

Open-source technologies

Batch and real-time pipelines

Data platform management

Data layer optimization

Apache Beam

dbt and data modeling

Tools

DataProc

DataFlow

Pub/Sub

BigQuery

BigTable

Cloud Spanner

CloudSQL

AlloyDB

Apache Airflow

dbt

Spark

Job description

IBM Consulting seeks an experienced Data Engineer to design, build, and maintain data pipelines on Google's Cloud ecosystem. You will leverage DataProc, DataFlow, Pub/Sub, BigQuery and BigTable to deliver scalable data solutions.

You will work with Apache Beam, Airflow, dbt, Spark/Python, and related tech to ensure robust batch and real-time processing, hosted from anywhere in the United States. This role emphasizes data accessibility, security, and performance.