GCP Data Architect

Ethereum Technologies

Dallas (TX)

On-site

USD 120,000 - 190,000

Full time

12 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Ethereum Technologies in Dallas, TX seeks an experienced IAM Data Architect to lead the end-to-end architecture for the IAM Data Modernization program. You will design metadata-driven ETL frameworks, build scalable ingestion patterns, and guide GCP/BigQuery integration across data lake zones.

You will provide technical leadership on PySpark, SQL, and data modeling while aligning CI/CD practices for data workloads. This on-site role offers significant impact on enterprise data modernization.

Qualifications

  • Hands-on experience with PySpark for ETL/ELT, data transformation, and performance optimization.
  • Solid understanding of distributed data processing concepts.
  • Strong experience designing data platforms on Google Cloud Platform (GCP).
  • Experience with Data Lakes, data warehousing, and large-scale migration programs.
  • Proven experience designing and implementing data lake architectures (Bronze/Silver/Gold or layered models).
  • Strong knowledge of Cloud Storage (GCS) design, including bucket layout, naming conventions, lifecycle policies, and access controls.
  • Expertise in partitioning strategies, backfills, and large-scale data organization.
  • Experience building batch and streaming ingestion pipelines using GCP-native services.
  • Knowledge of Pub/Sub-based streaming architectures, event schema design, and versioning.
  • Experience developing scalable batch and streaming pipelines using Dataflow (Apache Beam) and/or Spark (Dataproc).
  • Strong proficiency in BigQuery SQL, including query optimization, partitioning, clustering, and cost control.
  • Experience implementing CI/CD pipelines for data and analytics workloads.
  • Familiarity with Git-based source control, build automation, and deployment strategies.

Responsibilities

  • Own the end-to-end data architecture for the IAM Data Modernization program across OCP, S3, PostgreSQL metadata repository, and GCP / BigQuery future state architecture.
  • Define and guide implementation of a metadata-driven generic ETL framework supporting multiple source types including databases, files, APIs, MongoDB, Splunk, VDS, and Active Directory.
  • Architect reusable metadata models using MD_ static metadata tables and OPS_ operational/runtime tables for source configuration, pipeline definition, run tracking, audit logging, checkpointing, schema drift, and notifications.
  • Provide architecture direction for MVC-based orchestration, including repository layer, service layer, wrapper, controller, execution plan models, and standardized request/response contracts.
  • Design scalable ingestion patterns for full load, incremental load, watermark-based processing, CDC-style patterns, batch ingestion, API ingestion, and file-based ingestion.
  • Guide design of S3 data lake zones including Landing, Raw, Archive, and Error zones, with clear promotion, archival, replay, and error-handling patterns.
  • Define data quality, schema validation, schema drift detection, CLOB validation, checkpoint restartability, auditability, and operational resilience standards.
  • Provide technical leadership for GCP / BigQuery architecture, including S3 to BigQuery integration, schema mapping, dataset design, partitioning, clustering, performance optimization, and secure connectivity.
  • Review and guide CI/CD automation using GitHub Actions, JFrog, Harness, Liquibase, Helm, OCP deployments, and environment-specific configuration management.
  • Provide technical guidance to team on Python, PySpark, SQL, BigQuery, PostgreSQL, data modeling, performance tuning, and production-ready framework design.

Skills

PySpark ETL
Distributed data processing
GCP architecture
Data lake design
Partitioning strategies
Batch and streaming pipelines
Pub/Sub streaming
Dataflow / Dataproc
BigQuery SQL
CI/CD for analytics
Git workflows

Tools

Git
Google Cloud Dataflow
Google Cloud Dataproc
BigQuery
Cloud Storage (GCS)

Job description

US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other


Work Authorization:US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other


Job Description:

The IAM Data Modernization project involves migrating an on-premises SQL data warehouse to a target state Data Lake in GCP cloud environment.This modernization establishes a single source of truth for enterprise-wide data-driven decision-making. The IAM Data Modernization program is focused on migrating and modernizing enterprise IAM data platforms from on-premises systems into a scalable, metadata-driven modern data architecture.


Key Responsibilities:


  • Own the end-to-end data architecture for the IAM Data Modernization program across OCP, S3, PostgreSQL metadata repository, and GCP / BigQuery future state architecture.

  • Define and guide implementation of a metadata-driven generic ETL framework supporting multiple source types including databases, files, APIs, MongoDB, Splunk, VDS, and Active Directory.

  • Architect reusable metadata models using MD_ static metadata tables and OPS_ operational/runtime tables for source configuration, pipeline definition, run tracking, audit logging, checkpointing, schema drift, and notifications.

  • Provide architecture direction for MVC-based orchestration, including repository layer, service layer, wrapper, controller, execution plan models, and standardized request/response contracts.

  • Design scalable ingestion patterns for full load, incremental load, watermark-based processing, CDC-style patterns, batch ingestion, API ingestion, and file-based ingestion.

  • Guide design of S3 data lake zones including Landing, Raw, Archive, and Error zones, with clear promotion, archival, replay, and error-handling patterns.

  • Define data quality, schema validation, schema drift detection, CLOB validation, checkpoint restartability, auditability, and operational resilience standards.

  • Provide technical leadership for GCP / BigQuery architecture, including S3 to BigQuery integration, schema mapping, dataset design, partitioning, clustering, performance optimization, and secure connectivity.

  • Review and guide CI/CD automation using GitHub Actions, JFrog, Harness, Liquibase, Helm, OCP deployments, and environment-specific configuration management.

  • Provide technical guidance to team on Python, PySpark, SQL, BigQuery, PostgreSQL, data modeling, performance tuning, and production-ready framework design.


Required Skills:


  • Hands-on experience with PySpark for ETL/ELT, data transformation, and performance optimization

  • Solid understanding of distributed data processing concepts

  • Strong experience designing data platforms on Google Cloud Platform (GCP)

  • Experience with Data Lakes, data warehousing, and large-scale migration programs

  • Proven experience designing and implementing data lake architectures(e.g., Bronze/Silver/Gold or layered models).

  • Strong knowledge of Cloud Storage (GCS)design, including bucket layout, naming conventions, lifecycle policies, and access controls

  • Expertise in partitioning strategies, backfills, and large-scale data organization

  • Experience building batch and streaming ingestion pipelinesusing GCP-native services

  • Knowledge of Pub/Sub-based streaming architectures, event schema design, and versioning

  • Experience developing scalable batch and streaming pipelinesusing Dataflow (Apache Beam) and/or Spark (Dataproc)Strong proficiency in BigQuery SQL, including query optimization, partitioning, clustering, and cost control.Experience implementing CI/CD pipelinesfor data and analytics workloads

  • Familiarity with Git-based source control, build automation, and deployment strategies


We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

GCP Data Architect
GCP Data Architect

Quantum Technologies. LLC • Dallas (TX)

On-site
USD 150,000 - 190,000
GCP Data Architect
GCP Data Architect

Robotic Technologies Automation • Dallas (TX)

On-site
USD 140,000 - 170,000
GCP Data Architect
GCP Data Architect

CloudBigD • Dallas (TX)

On-site
USD 140,000 - 210,000
GCP Data Architect
GCP Data Architect

Valiant Technologies LLC • Dallas (TX)

On-site
USD 140,000 - 200,000
GCP Data Architect
GCP Data Architect

Robotics Technologies LLC • Dallas (TX)

On-site
USD 120,000 - 190,000
GCP Data Engineer
GCP Data Engineer

Ethereum Technologies LLC • Tampa (FL)

On-site
USD 110,000 - 150,000
GCP Data Engineer
GCP Data Engineer

Valiant Technologies LLC • Tampa (FL)

On-site
USD 110,000 - 165,000
GCP Data Engineer
GCP Data Engineer

Quantum Technologies LLC • Tampa (FL)

On-site
USD 120,000 - 180,000
Senior Data Architect
Senior Data Architect

Arkhya Tech. Inc. • Dallas (TX)

On-site
USD 120,000 - 150,000
GCP Architect
GCP Architect

Ethereum Technologies LLC • Memphis (TN)

On-site
USD 180,000 - 240,000