Real-Time Data Engineer, CDP on GCP (Hybrid)

TekWissen LLC; Atlanta (GA); Hybrid; USD 100,000 - 150,000; Part time

Job summary

TekWissen LLC is seeking a Data Engineer in Atlanta to design and operate real-time pipelines behind a Customer Data Platform. The role focuses on moving to Google Cloud, building low-latency data flows, and enabling APIs for AI agents across multiple data sources including clickstream, care, and network data.

The ideal candidate has extensive experience in Python, Databricks, PySpark, Delta Lake, and SQL, with hands-on work on BigQuery, Spanner, and Snowflake in hybrid cloud environments.

Qualifications

• 5+ years in data engineering with production real-time pipelines.
• Production data pipelines on Google Cloud with IAM, VPC Service Controls, and cost awareness.
• Streaming & NRT pipelines with Pub/Sub/Dataflow, Kafka, Event Hubs, or Spark Structured Streaming with exactly-once semantics.

Responsibilities

• Build and run near-real-time ingestion into Spanner and BigQuery with data contracts and exactly-once semantics.
• Implement CDC, SCD Type 2, and late-arriving data patterns at scale with Beam, SQL, and PySpark.
• Migrate Azure ETL to GCP using Databricks, Data Factory, and Snowflake; retire Cosmos DB/Redis/Event Hubs as needed.
• Develop customer knowledge graph pipelines and feed the GraphQL API and MCP data feeds on Cloud Run.
• Enforce data quality, lineage, IAM controls, and ship with Terraform, CI/CD, monitoring, and cost controls.

Skills

Python

Databricks

PySpark

Delta Lake

Snowflake

SQL

BigQuery

Education

Bachelor's degree in Computer Science or related field

Tools

Dataflow

Pub/Sub

BigQuery

Spanner

Cosmos DB

Redis

Event Hubs

Terraform

CI/CD

Cloud Run

Dataproc

GraphQL

Snowflake

Job description

TekWissen LLC is seeking a Data Engineer in Atlanta to design and operate real-time pipelines behind a Customer Data Platform. The role focuses on moving to Google Cloud, building low-latency data flows, and enabling APIs for AI agents across multiple data sources including clickstream, care, and network data.

The ideal candidate has extensive experience in Python, Databricks, PySpark, Delta Lake, and SQL, with hands-on work on BigQuery, Spanner, and Snowflake in hybrid cloud environments.