GCP Data Engineer: BigQuery & Dataflow Architect

Valiant Technologies LLC; Tampa (FL); On-site; USD 110,000 - 165,000; Full time

Job summary

Valiant Technologies LLC in Tampa, FL is seeking an experienced GCP Data Engineer to build scalable cloud-based data pipelines on Google Cloud Platform. You will work with large datasets, develop ETL/ELT workflows, optimize data warehouses, and support analytics and BI initiatives.

The role requires hands-on experience with BigQuery, Python, SQL, Dataflow, Airflow, Pub/Sub, and related GCP services, plus governance and security best practices.

Qualifications

• Experience with Google Cloud Platform (GCP) data services.
• Proficient in Python and SQL for data transformation.
• Strong BigQuery knowledge including SQL optimization and cost management.
• Experience with Cloud Dataflow and Apache Beam for pipelines.
• Familiarity with Cloud Composer (Airflow) for orchestration.
• Knowledge of Pub/Sub for real-time data ingestion.
• Experience with Dataproc, Spark, and batch processing.
• Understanding data modeling and data warehouse architecture.
• Experience with ETL/ELT development and data quality checks.
• Knowledge of IAM, encryption, and security best practices on GCP.
• CI/CD and infrastructure automation familiarity.

Responsibilities

• Design and implement scalable batch and streaming data pipelines on GCP.
• Build ingestion frameworks to load data from APIs, databases, files, and streaming sources into BigQuery.
• Develop data transformation logic using SQL, Python, Apache Beam, and Spark where appropriate.
• Create and maintain BigQuery datasets, tables, views, and optimized analytical data models.
• Orchestrate and monitor workflows using Cloud Composer and Airflow DAGs.
• Implement real-time processing solutions using Pub/Sub and Dataflow.
• Optimize query performance, storage usage, and cloud infrastructure costs.
• Build data quality checks, reconciliation processes, and pipeline failure recovery mechanisms.
• Integrate data from enterprise systems, cloud applications, and on-premises sources.
• Implement data access controls, governance, lineage, and security policies.
• Configure logging, monitoring, alerting, and troubleshooting for production pipelines.
• Collaborate with data architects, analysts, BI developers, and machine learning teams.
• Maintain technical documentation and support production releases.

Skills

Python

SQL

Data modeling

Tools

BigQuery

Cloud Dataflow

Apache Beam

Cloud Composer

Pub/Sub

Cloud Storage

Dataproc

Spark

Git

CI/CD

Infrastructure automation

Job description

Valiant Technologies LLC in Tampa, FL is seeking an experienced GCP Data Engineer to build scalable cloud-based data pipelines on Google Cloud Platform. You will work with large datasets, develop ETL/ELT workflows, optimize data warehouses, and support analytics and BI initiatives.

The role requires hands-on experience with BigQuery, Python, SQL, Dataflow, Airflow, Pub/Sub, and related GCP services, plus governance and security best practices.