Data Engineer - Real-Time Streaming (GCP/PySpark)

CloudBigD

Charlotte (NC)

On-site

USD 91,000 - 124,000

Full time

37 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

CloudBigD in Charlotte, NC is seeking a Data Engineer to design, build, and maintain scalable data pipelines using Python/Java, GCP, and PySpark. The role focuses on distributed processing, real-time streaming, REST APIs, and database integration with Oracle and S3.

You'll work with Spark and Flink in a Tier-1 vendor environment, collaborating with cross-functional teams to deliver robust data integration solutions.

Qualifications

  • Strong Python or Java development experience.
  • Solid data engineering and data pipeline development expertise.
  • Hands-on experience with GCP.
  • Practical experience with PySpark.
  • Experience building and deploying pipelines using Apache Spark.
  • Understanding of Apache Flink and distributed data systems.
  • Hands-on experience with real-time streaming architectures.
  • Experience working with REST APIs and database connectors.
  • Proficiency with relational databases such as Oracle.
  • Experience using Amazon S3 APIs for data storage and integration.
  • Prior Tier 1 vendor experience is mandatory.

Responsibilities

  • Design, develop, and deploy scalable data pipelines using Python, Java, and PySpark.
  • Build and maintain data processing solutions on Google Cloud Platform (GCP).
  • Work with distributed data processing frameworks such as Apache Spark and Apache Flink.
  • Develop and support real-time streaming data architectures.
  • Integrate data sources and services using REST APIs and database connectors.
  • Work with relational databases, including Oracle, for data storage and processing.
  • Implement data storage and integration solutions using Amazon S3 APIs.
  • Troubleshoot, optimize, and maintain data pipelines for performance, scalability, and reliability.
  • Collaborate with cross-functional teams to deliver data engineering solutions.

Skills

Python
Java
PySpark
GCP
REST APIs
Oracle
S3 APIs
Spark
Flink
Data pipelines

Tools

Google Cloud Platform
Apache Spark
Apache Flink
Amazon S3
Oracle Database

Job description

CloudBigD in Charlotte, NC is seeking a Data Engineer to design, build, and maintain scalable data pipelines using Python/Java, GCP, and PySpark. The role focuses on distributed processing, real-time streaming, REST APIs, and database integration with Oracle and S3.

You'll work with Spark and Flink in a Tier-1 vendor environment, collaborating with cross-functional teams to deliver robust data integration solutions.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Real-Time Data Engineer – Spark, Kafka & GCP
Real-Time Data Engineer – Spark, Kafka & GCP

Compunnel Inc. • Sunnyvale (CA)

On-site
USD 100,000 - 140,000
Senior Data Engineer
Senior Data Engineer

Compunnel Inc. • Sunnyvale (CA)

On-site
USD 100,000 - 140,000
Data Engineer - GCP, PySpark, Airflow & Kafka
Data Engineer - GCP, PySpark, Airflow & Kafka

Tata Consultancy Services Limited • Sunnyvale (CA)

On-site
USD 100,000 - 120,000
Discretionary Annual Incentive
Comprehensive Medical Coverage
Family Leave
+3
Data Engineer
Data Engineer

CloudBigD • Charlotte (NC)

On-site
USD 91,000 - 124,000
Senior Data Engineer - Real-Time Pipelines on GCP
Senior Data Engineer - Real-Time Pipelines on GCP

KANINI Software Solutions PTE Ltd. • Nashville (TN)

On-site
USD 110,000 - 170,000
Senior Data Engineer - Real-Time Pipelines (Spark/Kafka)
Senior Data Engineer - Real-Time Pipelines (Spark/Kafka)

UST • Sunnyvale (CA)

On-site
USD 72,000 - 108,000
Medical Insurance
401(k) with employer match
Paid time off
Real-Time Data Engineer – Kinesis, S3 & Oracle
Real-Time Data Engineer – Kinesis, S3 & Oracle

Datumtg • Charlotte (NC)

On-site
USD 110,000 - 150,000
Data Engineer - GCP, PySpark & BigQuery Pipelines
Data Engineer - GCP, PySpark & BigQuery Pipelines

Valiant Technologies LLC • Woodbridge Township (NJ)

On-site
USD 77,000 - 110,000
Data Engineer
Data Engineer

EdgeQ Technologies Private Limited • United States

On-site
USD 120,000 - 180,000
Senior Data Engineer: Real-Time Pipelines & Cloud Analytics
Senior Data Engineer: Real-Time Pipelines & Cloud Analytics

Cloudndata • United States

Hybrid
USD 140,000 - 180,000