Remote GCP Data Engineer — Build Scalable Pipelines

The Data Sherpas; Atlanta (GA); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Health coverage

Remote work options

GCP certification training

Job summary

The Data Sherpas is seeking a Google Cloud Data Engineer to design, develop, and manage scalable data pipelines and data infrastructure on Google Cloud Platform. You will ensure data availability, accuracy, and performance for business insights and ML models.

You will collaborate with data scientists and analysts, implement data transformations, and maintain data governance and security across environments.

Qualifications

• 3+ years in data engineering with 2+ years on GCP.
• Google Professional Data Engineer certification required.
• Proficient with BigQuery, Dataflow, Composer, Pub/Sub, Firestore, Functions.
• Experience with Apache Beam, Hadoop, Spark, or Flink.
• Proficient in Python, Java, or Scala.
• Strong SQL, data modeling, and query optimization.
• Experience with CI/CD and Git.
• Knowledge of data governance, security, and compliance.
• Ability to manage large-scale, real-time pipelines.
• Excellent problem solving, analytical and communication skills.

Responsibilities

• Design, build, and maintain scalable data pipelines on GCP.
• Develop ETL/ELT processes for large datasets.
• Optimize pipeline performance and reliability to meet SLAs.
• Ensure data processing and ingestion workflows are monitored.
• Design and implement data storage using BigQuery, Cloud Storage, Firestore.
• Optimize data structures and partitioning for cost efficiency.
• Develop data transformation processes using BigQuery, Apache Beam, Cloud Functions.
• Implement data quality checks and monitoring solutions.
• Integrate data from multiple sources including APIs and databases.
• Automate data ingestion, transformation, and exports using Composer and Functions.
• Collaborate with data scientists and analysts to meet business goals.
• Provide guidance on best practices for data governance and security.

Skills

Data engineering

Python

Java

Scala

SQL

Data modeling

Communication

Problem solving

Education

Bachelor’s degree in CS/related field

Master’s degree (preferred)

Google Professional Data Engineer certification

Tools

BigQuery

Cloud Dataflow

Cloud Composer

Cloud Pub/Sub

Firestore

Cloud Functions

Apache Beam

Hadoop

Spark

Flink

dbt

Git

Cloud Build

Job description

The Data Sherpas is seeking a Google Cloud Data Engineer to design, develop, and manage scalable data pipelines and data infrastructure on Google Cloud Platform. You will ensure data availability, accuracy, and performance for business insights and ML models.

You will collaborate with data scientists and analysts, implement data transformations, and maintain data governance and security across environments.