GCP Data Engineer — BigQuery & Data Pipelines

MACHINE LEARNING TECHNOLOGIES LLC; Tampa (FL); On-site; USD 110,000 - 150,000; Full time

Job summary

MACHINE LEARNING TECHNOLOGIES LLC is seeking an experienced GCP Data Engineer in Tampa to design and maintain scalable cloud-based data pipelines. You will work with large datasets, develop ETL/ELT workflows, and optimize data warehouses on Google Cloud Platform.

You will implement data processing with Python, SQL, Beam, Spark, and Airflow, and collaborate with analytics and BI teams to drive insights and governance across production pipelines.

Qualifications

• Extensive experience with GCP data services.
• Expertise in BigQuery optimization (partitioning, clustering, cost management).
• Proficient in Python and SQL for data processing.
• Hands-on with Dataflow and Apache Beam for ETL/ELT pipelines.
• Experience with Cloud Composer (Airflow) for orchestration.
• Knowledge of Pub/Sub for real-time data ingestion.
• Familiarity with Dataproc/Spark for batch processing.
• Strong data modeling and data warehouse architecture understanding.
• Implement data quality checks and transformations.
• Security practices: IAM, encryption, service accounts.
• Experience with Git, CI/CD, and IaC automation.

Responsibilities

• Design scalable batch and streaming data pipelines on GCP.
• Ingest data from APIs, databases, files, and streams into BigQuery.
• Develop data transformations using SQL, Python, Beam, and Spark.
• Create and optimize BigQuery datasets, tables, views and analytics models.
• Orchestrate workflows with Cloud Composer and Airflow DAGs.
• Implement real-time processing with Pub/Sub and Dataflow.
• Tune queries, manage storage, and optimize cloud costs.
• Build data quality checks and recovery mechanisms.
• Integrate data from enterprise systems and on-prem sources.
• Enforce data access controls, governance, and lineage.
• Configure monitoring, logging, and alerting for production pipelines.
• Collaborate with data architects, analysts, BI developers, and ML teams.
• Maintain technical documentation and support releases.

Skills

GCP data services

BigQuery

Python

SQL

Dataflow

Apache Beam

Cloud Composer

Pub/Sub

Dataproc

Spark

Data modeling

ETL/ELT

Security best practices

IAM

CI/CD

Infrastructure as code

Git

Tools

BigQuery

Apache Airflow

Apache Beam

Spark

Dataproc

Pub/Sub

Cloud Dataflow

Cloud Storage

IAM

Git

Job description

MACHINE LEARNING TECHNOLOGIES LLC is seeking an experienced GCP Data Engineer in Tampa to design and maintain scalable cloud-based data pipelines. You will work with large datasets, develop ETL/ELT workflows, and optimize data warehouses on Google Cloud Platform.

You will implement data processing with Python, SQL, Beam, Spark, and Airflow, and collaborate with analytics and BI teams to drive insights and governance across production pipelines.