Data Platform Engineer: Hadoop-to-GCP & Real-Time Streaming

TEKsystems; Charlotte (NC); Hybrid; USD 69,000 - 74,000; Full time

Benefits offered by this job

Medical, dental & vision

401(k) Retirement Plan

Life Insurance

Short and long-term disability

Health Savings Account (HSA)

Transportation benefits

Employee Assistance Program

Time Off / PTO

Hybrid work model

Job summary

TEKsystems is seeking a midlevel Data Engineer in Charlotte, NC, to build and deploy data pipelines across Hadoop and Google Cloud Platform. You will work with Spark-based processing, PySpark, Python, and real-time streaming using Kafka in a collaborative team.

The role includes migrating on-prem workloads to GCP and implementing data flows with BigQuery, Cloud Composer, Data Proc, and GCP Storage. Hybrid work in Charlotte, with a focus on scalable, production-ready solutions.

Qualifications

• Hands-on data engineering with Hadoop, PySpark and Python in production.
• Experience migrating on-prem workloads to GCP and designing scalable data pipelines.
• Ability to translate requirements into robust data architectures and deployments.

Responsibilities

• Build and support Spark-based processing pipelines in Hadoop using PySpark and Python.
• Develop and maintain real-time streaming pipelines using Kafka.
• Design and implement data flows using Spark Streaming and Flink.
• Migrate data and workloads from on-prem Hadoop to GCP.
• Work with BigQuery, Cloud Composer, Data Proc and GCP Cloud Storage.

Skills

Hadoop

PySpark

Python

Kafka

Spark Streaming

Flink

BigQuery

Cloud Composer

Data Proc

GCP

Tools

Google Cloud Platform

Job description

TEKsystems is seeking a midlevel Data Engineer in Charlotte, NC, to build and deploy data pipelines across Hadoop and Google Cloud Platform. You will work with Spark-based processing, PySpark, Python, and real-time streaming using Kafka in a collaborative team.

The role includes migrating on-prem workloads to GCP and implementing data flows with BigQuery, Cloud Composer, Data Proc, and GCP Storage. Hybrid work in Charlotte, with a focus on scalable, production-ready solutions.