Remote Google Data Engineer - BigQuery & Dataflow Pro

IBM; New York (NY); Remote; USD 120,000 - 160,000; Full time

Job summary

IBM Consulting seeks a seasoned Data Engineer specializing in Google's data platforms to design, build, and maintain scalable pipelines on Google Cloud. You will leverage open-source tools such as Apache Beam, Airflow, dbt, Spark and Python to deliver batch and real-time data solutions for data warehouses and lakes, across client engagements.

Remote-friendly role based in the United States, with opportunities to collaborate globally.

Qualifications

• Deep expertise designing data platforms on Google Cloud (DataProc, DataFlow, Pub/Sub, BigQuery, BigTable, Cloud Spanner, CloudSQL, AlloyDB).
• Proficiency with Apache Beam, Airflow, dbt, Spark/Python or Spark/Scala and integrating with Google Cloud services.
• Experience building batch and real-time data pipelines for Data Warehouse and Data Lake.
• Experience scheduling and managing data platforms using Cloud Scheduler and Cloud Composer (Airflow).
• Ability to design and optimize data layers for migration and processing.

Responsibilities

• Design batch and real-time data pipelines for Data Warehouse and Data Lake using Google Cloud services.
• Build and maintain data engineering solutions with Google Cloud Storage, BigTable, BigQuery, DataProc, Spark/Hadoop, DataFlow with Apache Beam or Python, and dbt/Spark integrations.
• Manage data platforms with Google Cloud Scheduler and Cloud Composer for reliable operations.
• Optimize data layer for efficient data migration and processing.
• Ensure scalability and efficiency of pipelines to meet client needs.

Skills

Google Data Platforms

Open-Source Tech

Batch & Real-Time Pipelines

Data Platform Management

Data Layer Optimization

Tools

DataProc

DataFlow

PubSub

BigQuery

BigTable

Cloud Spanner

CloudSQL

AlloyDB

Apache Beam

Airflow

dbt

Spark

Python

Scala

Job description

IBM Consulting seeks a seasoned Data Engineer specializing in Google's data platforms to design, build, and maintain scalable pipelines on Google Cloud. You will leverage open-source tools such as Apache Beam, Airflow, dbt, Spark and Python to deliver batch and real-time data solutions for data warehouses and lakes, across client engagements.

Remote-friendly role based in the United States, with opportunities to collaborate globally.