GCP Data Engineer

MACHINE LEARNING TECHNOLOGIES LLC; Tampa (FL); On-site; USD 110,000 - 150,000; Full time

Job summary

MACHINE LEARNING TECHNOLOGIES LLC is seeking an experienced GCP Data Engineer in Tampa to design and maintain scalable cloud-based data pipelines. You will work with large datasets, develop ETL/ELT workflows, and optimize data warehouses on Google Cloud Platform.

You will implement data processing with Python, SQL, Beam, Spark, and Airflow, and collaborate with analytics and BI teams to drive insights and governance across production pipelines.

Qualifications

• Extensive experience with GCP data services.
• Expertise in BigQuery optimization (partitioning, clustering, cost management).
• Proficient in Python and SQL for data processing.
• Hands-on with Dataflow and Apache Beam for ETL/ELT pipelines.
• Experience with Cloud Composer (Airflow) for orchestration.
• Knowledge of Pub/Sub for real-time data ingestion.
• Familiarity with Dataproc/Spark for batch processing.
• Strong data modeling and data warehouse architecture understanding.
• Implement data quality checks and transformations.
• Security practices: IAM, encryption, service accounts.
• Experience with Git, CI/CD, and IaC automation.

Responsibilities

• Design scalable batch and streaming data pipelines on GCP.
• Ingest data from APIs, databases, files, and streams into BigQuery.
• Develop data transformations using SQL, Python, Beam, and Spark.
• Create and optimize BigQuery datasets, tables, views and analytics models.
• Orchestrate workflows with Cloud Composer and Airflow DAGs.
• Implement real-time processing with Pub/Sub and Dataflow.
• Tune queries, manage storage, and optimize cloud costs.
• Build data quality checks and recovery mechanisms.
• Integrate data from enterprise systems and on-prem sources.
• Enforce data access controls, governance, and lineage.
• Configure monitoring, logging, and alerting for production pipelines.
• Collaborate with data architects, analysts, BI developers, and ML teams.
• Maintain technical documentation and support releases.

Skills

GCP data services

BigQuery

Python

SQL

Dataflow

Apache Beam

Cloud Composer

Pub/Sub

Dataproc

Spark

Data modeling

ETL/ELT

Security best practices

IAM

CI/CD

Infrastructure as code

Git

Tools

BigQuery

Apache Airflow

Apache Beam

Spark

Dataproc

Pub/Sub

Cloud Dataflow

Cloud Storage

IAM

Git

Job description

Job Description

• Job ID: J53713
• Job Title: GCP Data Engineer
• Location: Tampa, FL
• Duration: 24 Months + Extension
• Domain Exposure: Aerospace/ Airlines
• Work Authorization: US Citizen,Green Card,OPT-EAD,CPT,H-1B,H4-EAD,L2-EAD,GC-EAD
• Client: To Be Discussed Later
• Employment Type: W-2, 1099, C2C

We are looking for an experienced GCP Data Engineer to build and maintain scalable cloud-based data pipelines and data processing solutions on Google Cloud Platform. The candidate will work with large datasets, develop ETL/ELT workflows, optimize data warehouses, and support analytics and business intelligence initiatives.

Technical Skills

• Strong experience with Google Cloud Platform (GCP) data services.
• Hands-on experience with BigQuery, including SQL optimization, partitioning, clustering, and cost management.
• Strong programming skills in Python and advanced SQL.
• Experience developing data pipelines using Cloud Dataflow and Apache Beam.
• Experience with Cloud Composer (Apache Airflow) for workflow orchestration.
• Knowledge of Pub/Sub for real-time data ingestion and event-driven processing.
• Experience with Cloud Storage, Dataproc, and Spark for batch processing.
• Understanding of data modeling, dimensional modeling, and data warehouse architecture.
• Experience with ETL/ELT development, data transformation, and data quality validation.
• Knowledge of IAM, service accounts, encryption, and GCP security best practices.
• Familiarity with Git, CI/CD, and infrastructure automation.

Roles and Responsibilities

• Design and implement scalable batch and streaming data pipelines on GCP.
• Build ingestion frameworks to load data from APIs, databases, files, and streaming sources into BigQuery.
• Develop data transformation logic using SQL, Python, Apache Beam, and Spark where appropriate.
• Create and maintain BigQuery datasets, tables, views, and optimized analytical data models.
• Orchestrate and monitor workflows using Cloud Composer and Airflow DAGs.
• Implement real-time processing solutions using Pub/Sub and Dataflow.
• Optimize query performance, storage usage, and cloud infrastructure costs.
• Build data quality checks, reconciliation processes, and pipeline failure recovery mechanisms.
• Integrate data from enterprise systems, cloud applications, and on-premises sources.
• Implement data access controls, governance, lineage, and security policies.
• Configure logging, monitoring, alerting, and troubleshooting for production pipelines.
• Collaborate with data architects, analysts, BI developers, and machine learning teams.
• Maintain technical documentation and support production releases.

Equal Opportunity Employer

We are an equal opportunity employer.

All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs.

We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.