GCP Data Engineer: Scalable Pipelines & BigQuery

Ethereum Technologies; Tampa (FL); On-site; USD 110,000 - 150,000; Full time

Job summary

Ethereum Technologies in Tampa, FL seeks an experienced GCP Data Engineer to design and maintain scalable data pipelines and analytics-ready data models on Google Cloud Platform. The role focuses on BigQuery optimization, SQL tuning, and cost control, while building ETL/ELT workflows with Dataflow, Beam, and Airflow.

You will load data from APIs, databases, and streaming sources, orchestrate pipelines with Cloud Composer, and implement real-time processing using Pub/Sub.

Qualifications

• Strong experience with Google Cloud Platform data services.
• Hands-on experience with BigQuery, including SQL optimization, partitioning, clustering, and cost management.
• Strong programming skills in Python and advanced SQL.
• Experience developing data pipelines using Cloud Dataflow and Apache Beam.
• Experience with Cloud Composer (Apache Airflow) for workflow orchestration.
• Knowledge of Pub/Sub for real-time data ingestion and event-driven processing.
• Experience with Cloud Storage, Dataproc, and Spark for batch processing.
• Understanding of data modeling, dimensional modeling, and data warehouse architecture.
• Experience with ETL/ELT development, data transformation, and data quality validation.
• Knowledge of IAM, service accounts, encryption, and GCP security best practices.
• Familiarity with Git, CI/CD, and infrastructure automation.

Responsibilities

• Design and implement scalable batch and streaming data pipelines on GCP.
• Build ingestion frameworks to load data from APIs, databases, files, and streaming sources into BigQuery.
• Develop data transformation logic using SQL, Python, Apache Beam, and Spark where appropriate.
• Create and maintain BigQuery datasets, tables, views, and optimized analytical data models.
• Orchestrate and monitor workflows using Cloud Composer and Airflow DAGs.
• Implement real-time processing solutions using Pub/Sub and Dataflow.
• Optimize query performance, storage usage, and cloud infrastructure costs.
• Build data quality checks, reconciliation processes, and pipeline failure recovery mechanisms.
• Integrate data from enterprise systems, cloud applications, and on-premises sources.
• Implement data access controls, governance, lineage, and security policies.
• Configure logging, monitoring, alerting, and troubleshooting for production pipelines.
• Collaborate with data architects, analysts, BI developers, and machine learning teams.
• Maintain technical documentation and support production releases.

Skills

GCP Data Services

BigQuery expertise

Python programming

Advanced SQL

ETL/ELT development

Data modeling

Data warehousing

Tools

BigQuery

Cloud Dataflow

Apache Beam

Cloud Composer (Airflow)

Pub/Sub

Cloud Storage

Dataproc

Spark

Git

CI/CD

Infrastructure automation

Job description

Ethereum Technologies in Tampa, FL seeks an experienced GCP Data Engineer to design and maintain scalable data pipelines and analytics-ready data models on Google Cloud Platform. The role focuses on BigQuery optimization, SQL tuning, and cost control, while building ETL/ELT workflows with Dataflow, Beam, and Airflow.

You will load data from APIs, databases, and streaming sources, orchestrate pipelines with Cloud Composer, and implement real-time processing using Pub/Sub.