Remote Google Cloud Data Engineer – Pipelines & BigQuery

IBM; New York (NY); Remote; USD 120,000 - 180,000; Full time

Job summary

IBM Consulting seeks a Data Engineer specializing in Google Cloud to design, build, and maintain data pipelines across batch and real-time workloads. You will leverage DataProc, DataFlow, Pub/Sub, BigQuery, and related services to deliver scalable data solutions.

You will collaborate with cross-functional teams to optimize data architectures, ensure data quality, and align initiatives with business goals. Remote US work supported.

Qualifications

• Google Cloud Ecosystem Expertise: experience designing, building, and maintaining data engineering solutions on Google Cloud services including DataProc, DataFlow, Pub/Sub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB.
• Data Pipeline Development: develop and manage batch and real-time data pipelines for Data Warehouse and Data Lake using Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and DataFlow with Apache Beam or Python.
• Open Source Technologies: experience with Apache Airflow, dbt, Spark/Python, or Spark/Scala in data engineering solutions.
• Data Platform Management: scheduling and managing the data platform using Google Cloud Scheduler and Cloud Composer (Airflow).
• Data Engineering Tools: using various tools and technologies to develop and manage data solutions on Google Cloud.

Responsibilities

• Design Data Pipelines: design and build pipelines using Google DataProc, DataFlow, Pub/Sub, BigQuery, and other Google services.
• Develop Data Engineering Solutions: utilize Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and DataFlow with Apache Beam or Python to develop data engineering solutions.
• Manage Data Platforms: schedule and manage the data platform using Google Cloud Scheduler and Cloud Composer (Airflow).
• Ensure Data Quality: develop pipelines for Data Warehouse and Data Lake delivering high-quality data.
• Optimize Data Solutions: collaborate with stakeholders to align data solutions with business objectives.

Skills

Google Cloud

Data Pipelines

Open Source Technologies

Data Platform Management

Data Engineering Tools

Tools

BigQuery

DataProc

DataFlow

Pub/Sub

Cloud Composer

Apache Airflow

dbt

Job description

IBM Consulting seeks a Data Engineer specializing in Google Cloud to design, build, and maintain data pipelines across batch and real-time workloads. You will leverage DataProc, DataFlow, Pub/Sub, BigQuery, and related services to deliver scalable data solutions.

You will collaborate with cross-functional teams to optimize data architectures, ensure data quality, and align initiatives with business goals. Remote US work supported.