GCP Data Architect: Data Lake & BigQuery Lead

Quantum Technologies. LLC

Dallas (TX)

On-site

USD 150,000 - 190,000

Full time

14 hours ago
Be an early applicant

Get a reply from this recruiter

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Quantum Technologies. LLC seeks a senior data architect to lead the IAM Data Modernization effort, migrating on-premises data to a GCP-based Data Lake with BigQuery as the future state.

The role focuses on designing metadata-driven ETL, scalable ingestion, and robust data quality. You will guide architecture across MVC layers, CI/CD pipelines, and secure cloud connectivity, mentoring the team in PySpark, SQL, and Dataflow.

Qualifications

  • Hands-on PySpark ETL/ELT experience and performance tuning.
  • Strong grasp of distributed data processing concepts.
  • Extensive experience with GCP data services (BigQuery, Cloud Storage, Dataflow).
  • Proven ability to design scalable data lake architectures with layered zones.

Responsibilities

  • Own end-to-end data architecture for IAM Data Modernization across OCP, S3, PostgreSQL metadata repository, and GCP / BigQuery future state architecture.
  • Define and guide a metadata-driven ETL framework for databases, files, APIs, MongoDB, Splunk, VDS, and Active Directory.
  • Architect reusable metadata models for source config, pipelines, run tracking, audit logs, drift checks, and notifications.
  • Provide architecture direction for MVC-based orchestration and standardized request/response contracts.
  • Design scalable ingestion patterns: full load, incremental load, watermark-based processing, CDC, batch, API, and file-based ingestion.
  • Guide design of S3 data lake zones including Landing, Raw, Archive, and Error zones, with promotion, archival, replay, and error-handling patterns.
  • Define data quality, schema validation, schema drift detection, checkpoint restartability, auditability, and resilience standards.
  • Provide technical leadership for GCP / BigQuery architecture, including S3 to BigQuery integration, dataset design, partitioning, clustering, and secure connectivity.
  • Review CI/CD automation with GitHub Actions, JFrog, Harness, Liquibase, Helm, OCP deployments.
  • Provide guidance to team on Python, PySpark, SQL, BigQuery, PostgreSQL, data modeling, and performance tuning.

Skills

PySpark
Distributed data processing
GCP
Data lakes
Data lake architectures
Cloud Storage design
Partitioning strategies
Batch & streaming ingestion
Pub/Sub architectures
BigQuery SQL

Tools

Git
Dataflow
Dataproc
Liquibase
Helm
JFrog

Job description

Quantum Technologies. LLC seeks a senior data architect to lead the IAM Data Modernization effort, migrating on-premises data to a GCP-based Data Lake with BigQuery as the future state.

The role focuses on designing metadata-driven ETL, scalable ingestion, and robust data quality. You will guide architecture across MVC layers, CI/CD pipelines, and secure cloud connectivity, mentoring the team in PySpark, SQL, and Dataflow.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

GCP Data Architect: Modern Data Lake & BigQuery Lead
GCP Data Architect: Modern Data Lake & BigQuery Lead

CloudBigD • Dallas (TX)

On-site
USD 140,000 - 210,000
GCP Data Architect: Data Lake & BigQuery Lead
GCP Data Architect: Data Lake & BigQuery Lead

Robotic Technologies Automation • Dallas (TX)

On-site
USD 140,000 - 170,000
GCP Data Architect for Modern Data Lake & BigQuery
GCP Data Architect for Modern Data Lake & BigQuery

Ethereum Technologies • Dallas (TX)

On-site
USD 120,000 - 190,000
GCP Data Architect: IAM & Data Lake Modernization
GCP Data Architect: IAM & Data Lake Modernization

Valiant Technologies LLC • Dallas (TX)

On-site
USD 140,000 - 200,000
GCP Data Lake Architect & BigQuery Expert
GCP Data Lake Architect & BigQuery Expert

Robotics Technologies LLC • Dallas (TX)

On-site
USD 120,000 - 190,000
Onsite GCP Data Engineer & Architect — BigQuery, Dataflow
Onsite GCP Data Engineer & Architect — BigQuery, Dataflow

Quantum Technologies. LLC • Atlanta (GA)

Hybrid
USD 140,000 - 210,000
Data Engineer - GCP Pipeline & BigQuery Expert
Data Engineer - GCP Pipeline & BigQuery Expert

Quantum Technologies. LLC • Woodbridge Township (NJ)

On-site
USD 90,000 - 130,000
GCP Solutions Architect: Enterprise Modernization Lead
GCP Solutions Architect: Enterprise Modernization Lead

Quantum Technologies LLC • Memphis (TN)

On-site
USD 180,000 - 260,000
GCP Data Architect
GCP Data Architect

Robotic Technologies Automation • Dallas (TX)

On-site
USD 140,000 - 170,000
GCP Data Architect
GCP Data Architect

Valiant Technologies LLC • Dallas (TX)

On-site
USD 140,000 - 200,000