Data Engineer

GTECH LLC; Atlanta (GA); Hybrid; USD 96,000 - 138,000; Full time

Job summary

GTECH LLC in Frisco, TX / Atlanta, GA / Bellevue, WA is seeking a Senior Data Engineer with hands-on GCP, real-time pipelines, and Cloud Spanner.

You will build real-time data pipelines with Pub/Sub and Dataflow, optimize Spanner and BigQuery for high-volume ingestion, and implement CI/CD, monitoring, and data quality.

Qualifications

• 5+ years of Data Engineering experience, including 2+ years in production real-time pipelines.
• Mandatory hands-on experience with Google Cloud Spanner, including schema design, indexing, and data ingestion.
• Strong experience with GCP Pub/Sub, Dataflow (Apache Beam), and BigQuery.
• Expertise in SQL, Python, PySpark, Kafka/Spark Streaming, CDC, and SCD Type 2.
• Experience with Databricks, Delta Lake, Snowflake, and Azure data services.
• Knowledge of Terraform, CI/CD, and Airflow/Cloud Composer.

Responsibilities

• Build real-time data pipelines using GCP Pub/Sub and Dataflow (Apache Beam).
• Design and optimize Google Cloud Spanner and BigQuery for high-volume data ingestion and low-latency data access.
• Develop streaming pipelines using Kafka, Spark Structured Streaming, CDC, and PySpark.
• Implement data quality, monitoring, security, and CI/CD best practices.

Skills

Real-time pipelines

Google Cloud Platform

SQL

Python

PySpark

Kafka

Spark Streaming

CDC

SCD Type 2

Databricks

Delta Lake

Snowflake

Azure data services

Terraform

CI/CD

Airflow/Cloud Composer

Tools

Pub/Sub

Dataflow (Apache Beam)

Google Cloud Spanner

BigQuery

Kafka

Spark Structured Streaming

Databricks

Delta Lake

Snowflake

Azure data services

Terraform

Airflow

Cloud Composer

Job description

6 Months Contract

Hybrid Role - Frisco, TX | Atlanta, GA | Bellevue, WA

Video Interview

We are looking for a Senior Data Engineer with strong hands-on experience in GCP, real-time data pipelines, Cloud Spanner.

Key Responsibilities:

• Build real-time data pipelines using GCP Pub/Sub and Dataflow (Apache Beam).
• Design and optimize Google Cloud Spanner and BigQuery for high-volume data ingestion and low-latency data access.
• Develop streaming pipelines using Kafka, Spark Structured Streaming, CDC, and PySpark.
• Implement data quality, monitoring, security, and CI/CD best practices.

Required Skills:

• 5+ years of Data Engineering experience, including 2+ years in production real-time pipelines.
• Mandatory hands-on experience with Google Cloud Spanner, including schema design, indexing, and data ingestion.
• Strong experience with GCP Pub/Sub, Dataflow (Apache Beam), and BigQuery.
• Expertise in SQL, Python, PySpark, Kafka/Spark Streaming, CDC, and SCD Type 2.
• Experience with Databricks, Delta Lake, Snowflake, and Azure data services.
• Knowledge of Terraform, CI/CD, and Airflow/Cloud Composer.