Junior GCP Data Engineer - Real-Time Pipelines & Hadoop

TEKsystems; Irving (TX); Hybrid; USD 52,000 - 55,000; Full time

Benefits offered by this job

Medical, dental & vision

401(k) Retirement Plan

Life Insurance

Short and long-term disability

Health Spending Account (HSA)

Job summary

TEKsystems in Irving, TX is seeking a Jr-Mid Level GCP Data Engineer to support a large-scale migration from on-prem Hadoop to Google Cloud Platform. The role emphasizes hands-on data engineering using PySpark, Python, and real-time streaming with Kafka.

You will design, develop, and deploy data pipelines across Hadoop and GCP, implement data flows with Spark Streaming and Flink, and work with BigQuery, Cloud Composer, Data Proc, and Cloud Storage in a hybrid, W2 environment.

Qualifications

• 3+ years of experience in Data Engineering including hands on experience with Hadoop and Google Cloud Data Solutions.
• Experience with GCP Data Services: BigQuery, Cloud Storage, Data Proc, Cloud Composer; migration from on-prem to GCP.
• 2+ years of real time processing with Kafka, Flink, and Spark Streaming
• 3+ Years of data lake experience with Python

Responsibilities

• Design, develop, and deploy data pipelines across Hadoop and GCP environments.
• Build and maintain Spark-based processing pipelines in Hadoop using PySpark and Python.
• Develop real-time streaming pipelines using Kafka, Spark Streaming, and Flink.
• Migrate data and workloads from on-prem Hadoop to GCP; work with BigQuery, Cloud Composer, Data Proc, and Cloud Storage.

Skills

Data Engineering

Hadoop

GCP Data Solutions

BigQuery

Cloud Storage

Data Proc

Cloud Composer

Kafka

Flink

Spark Streaming

Python

Tools

PySpark

Spark

Job description

TEKsystems in Irving, TX is seeking a Jr-Mid Level GCP Data Engineer to support a large-scale migration from on-prem Hadoop to Google Cloud Platform. The role emphasizes hands-on data engineering using PySpark, Python, and real-time streaming with Kafka.

You will design, develop, and deploy data pipelines across Hadoop and GCP, implement data flows with Spark Streaming and Flink, and work with BigQuery, Cloud Composer, Data Proc, and Cloud Storage in a hybrid, W2 environment.