Data Engineer

TekWissen LLC

Atlanta (GA)

Hybrid

USD 100,000 - 150,000

Part time

10 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

TekWissen LLC is seeking a Data Engineer in Atlanta to design and operate real-time pipelines behind a Customer Data Platform. The role focuses on moving to Google Cloud, building low-latency data flows, and enabling APIs for AI agents across multiple data sources including clickstream, care, and network data.

The ideal candidate has extensive experience in Python, Databricks, PySpark, Delta Lake, and SQL, with hands-on work on BigQuery, Spanner, and Snowflake in hybrid cloud environments.

Qualifications

5+ years in data engineering with production real-time pipelines.
Production data pipelines on Google Cloud with IAM, VPC Service Controls, and cost awareness.
Streaming & NRT pipelines with Pub/Sub/Dataflow, Kafka, Event Hubs, or Spark Structured Streaming with exactly-once semantics.

Responsibilities

Build and run near-real-time ingestion into Spanner and BigQuery with data contracts and exactly-once semantics.
Implement CDC, SCD Type 2, and late-arriving data patterns at scale with Beam, SQL, and PySpark.
Migrate Azure ETL to GCP using Databricks, Data Factory, and Snowflake; retire Cosmos DB/Redis/Event Hubs as needed.
Develop customer knowledge graph pipelines and feed the GraphQL API and MCP data feeds on Cloud Run.
Enforce data quality, lineage, IAM controls, and ship with Terraform, CI/CD, monitoring, and cost controls.

Skills

Python

Databricks

PySpark

Delta Lake

Snowflake

SQL

BigQuery

Education

Bachelor's degree in Computer Science or related field

Tools

Dataflow

Pub/Sub

BigQuery

Spanner

Cosmos DB

Redis

Event Hubs

Terraform

CI/CD

Cloud Run

Dataproc

GraphQL

Snowflake

Job description

Overview

TekWissen is a global workforce management provider headquartered in Ann Arbor, Michigan that offers strategic talent solutions to our clients world-wide. Our client provider of digital technology and transformation, information technology and services

Job Title

Data Engineer

Location

Atlanta, GA or Frisco, TX (only locals)

Duration

6 Months

Job Type

Temporary Assignment

Work Type

Hybrid

Job Summary

a { text-decoration: none; color: #464feb; } tr th, tr td { border: 1px solid #e6e6e6; } tr th { background-color: #f5f5f5; }

The Data Engineer, CDP builds and operates the real-time pipelines behind client's Customer Data Platform - the system of record for 100M+ customer profiles, now moving to Google Cloud to deliver real-time customer context, near-real-time signals, and sub-50ms API access.Any agent, store associate, or digital channel should see a customer's prior conversation and continue from there. You will build how clickstream, care, store, and network data flows into Cloud Spanner (low-latency store), BigQuery (analytics), and the knowledge graph behind GraphQL and MCP servers - while migrating and reusing proven ETL from the Azure estate (Databricks, Snowflake, Data Factory, Event Hubs, Cosmos DB).

Responsibilities

Real-Time Pipeline Engineering

Build and run NRT ingestion with Pub/Sub and Dataflow (Beam) into Spanner and BigQuery, with data contracts, schema evolution, exactly-once semantics, dead-letter handling, and replay
Implement Spanner loads and change streams for current-state customer profiles, tuning keys, batching, and hotspots to hold sub-50ms reads

Clickstream, CDC & Data Modeling

Implement reusable CDC, SCD Type 2, and late-arriving-data patterns in Beam, SQL, and PySpark; build clickstream ingestion at scale
Partner with NBA and T-Engage to cut signal latency to sub-five minutes; build BigQuery Spanner sync and reverse ETL

Azure-to-GCP Migration

Move data from Azure to GCP with STS and BigQuery Spanner sync, reusing Databricks, Data Factory, and Snowflake ETL; execute migration or retirement of Cosmos DB, Redis, Event Hubs, and Functions with dual-run, reconciliation, and rollback

Knowledge Graph, API & MCP Data Feeds

Build the pipelines that populate the customer knowledge graph on Spanner (entity and identity resolution, Customer 360) and feed the sub-50ms GraphQL API and the MCP servers on Cloud Run used by AI agents

Quality, Operations & Delivery

Implement data quality, lineage, and access controls (Dataplex, Unity Catalog, TISS-310) and IAM/VPC-SC/CMEK; ship on Cloud Run, Dataproc, Dataflow, and Composer with Terraform, CI/CD, monitoring, SLOs, and cost tuning
Own pipeline on-call and runbooks, join design reviews, mentor junior engineers, and partner with product, analytics, and AI/ML teams

Required Experience

Overall (5+ years): Bachelor's in Computer Science or related field (or equivalent); 5+ years in data engineering, 2+ building real-time pipelines in production.
Google Cloud data platform (3+ years): Production data pipelines on GCP, including IAM, VPC Service Controls, and cost awareness.
Streaming & NRT pipelines (3+ years): Pub/Sub, Dataflow (Beam), Kafka, Event Hubs, or Spark Structured Streaming with exactly-once semantics, watermarking, and clickstream ingestion at scale.
BigQuery (3+ years): Partitioning and clustering, reservations and cost control, Spanner federation, and BigQuery-to-Spanner (reverse ETL) data movement.
Cloud Spanner (1+ year): Schema and primary key design, indexes, hotspot avoidance, change streams, and bulk and streaming loads.
Data modeling & ETL (4+ years): Dimensional, Data Vault, and medallion models; idempotent ETL/ELT (CDC, SCD Type 2); expert SQL, Python, and advanced PySpark.
Databricks, Spark & Delta Lake (3+ years): Spark tuning, Delta Lake, Delta Live Tables, and Unity Catalog, to reuse and migrate existing ETL.
Azure & Snowflake (2+ years): Data Factory, Event Hubs, Cosmos DB, Redis, and Functions, plus Snowflake, to support a hybrid-cloud migration.
Serving & APIs (1+ year): Building data feeds for REST/gRPC or GraphQL services with p99 targets under 50ms; exposure to MCP, LLM/agent, or Vertex AI data access a plus.
Platform engineering (3+ years): Terraform/IaC, CI/CD, Cloud Run, Dataproc, and Composer or Airflow; observability and SLOs.
Testing & data quality (2+ years): Unit, integration, and data-quality testing for pipelines; reconciliation, lineage, and Dataplex or similar tools.
Collaboration (2+ years): Working with product, analytics, and AI/ML teams; clearly explaining design tradeoffs; mentoring junior engineers.