Senior GCP Data Engineer

KTek Resourcing LLC; United States; Sur place; USD 120 000 - 180 000; Plein temps

Résumé du poste

KTek Resourcing LLC seeks an experienced GCP Data Engineer to design, develop, and maintain scalable data solutions on Google Cloud Platform. You will work with BigQuery, Cloud Storage, Dataflow, Pub/Sub, Cloud Composer, and Python to build modern data pipelines for large-scale datasets.

The role focuses on ETL/ELT development, data modeling, and data warehousing with emphasis on performance, cost optimization, and reliability across batch and streaming workloads.

Qualifications

• Bachelor's degree in computer science, engineering, information technology, or related field.
• 5+ years of experience in data engineering with strong hands-on GCP experience.
• Strong understanding of distributed data processing and cloud-native data architecture.
• Excellent problem-solving, communication, and collaboration skills.

Responsabilités

• Design and develop end-to-end data pipelines using GCP data services.
• Build scalable ingestion and transformation pipelines using Cloud Dataflow / Apache Beam.
• Develop complex SQL queries, stored procedures, and data transformations in BigQuery.
• Design and optimize BigQuery data warehouses, including partitioning, clustering, performance tuning, and cost optimization.
• Implement batch and real-time data processing using Dataflow, Pub/Sub, and BigQuery.
• Develop data ingestion solutions using Google Cloud Storage (GCS) and other cloud data sources.
• Build and manage workflow orchestration using Cloud Composer / Apache Airflow.
• Develop Python-based ETL/ELT frameworks and reusable data engineering components.
• Integrate data from APIs, databases, files, streaming platforms, and enterprise applications.
• Implement data quality, validation, monitoring, logging, and error-handling frameworks.

Connaissances

Python

SQL

ETL/ELT

Batch processing

Streaming

Data modeling

Data warehousing

Apache Beam

Airflow

Spark

Formation

Bachelor's degree

Outils

BigQuery

Cloud Spanner

PostgreSQL/MySQL

Google Cloud Storage

Cloud Composer

Dataflow

Apache Airflow

Cloud IAM

GCS Computer Networking

Description du poste

We are seeking an experienced GCP Data Engineer to design, develop, and maintain scalable data engineering solutions on Google Cloud Platform (GCP). The ideal candidate will have strong hands-on experience with BigQuery, Cloud Storage, Dataflow, Pub/Sub, Cloud Composer, Cloud Functions, Python, and SQL, along with experience building modern data pipelines and working with large-scale structured and unstructured datasets.

Key Responsibilities

• Design and develop end-to-end data pipelines using GCP data services.
• Build scalable ingestion and transformation pipelines using Cloud Dataflow / Apache Beam.
• Develop complex SQL queries, stored procedures, and data transformations in BigQuery.
• Design and optimize BigQuery data warehouses, including partitioning, clustering, performance tuning, and cost optimization.
• Implement batch and real-time data processing using Dataflow, Pub/Sub, and BigQuery.
• Develop data ingestion solutions using Google Cloud Storage (GCS) and other cloud data sources.
• Build and manage workflow orchestration using Cloud Composer / Apache Airflow.
• Develop Python-based ETL/ELT frameworks and reusable data engineering components.
• Integrate data from APIs, databases, files, streaming platforms, and enterprise applications.
• Implement data quality, validation, monitoring, logging, and error-handling frameworks.

Required Technical Skills

• Programming: Python, SQL
• Data Engineering: ETL/ELT, batch processing, streaming, data modeling, data warehousing
• Apache: Beam, Spark, Airflow
• Databases: BigQuery, Cloud Spanner, PostgreSQL/MySQL or other relational databases
• Strong understanding of GCP IAM, networking, security, monitoring, and cost optimization
• Preferred / Nice-to-Have Skills
• Experience with Cloud Spanner architecture and design
• Experience with Spanner Graph / Graph Database / Graph Data
• GCP Professional Data Engineer certification

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field.
• 5+ years of experience in data engineering, with strong hands-on GCP experience.
• Strong understanding of distributed data processing and cloud-native data architecture.
• Excellent problem-solving, communication, and collaboration skills.