Senior GCP Data Engineer/Data Architect Consultant

iScale Solutions; United States; Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

WFH or Hybrid

Competitive salary

Flexible schedule

Job summary

iScale Solutions is seeking a data engineering expert to shape and implement cloud-native analytics on Google Cloud. You will design a Bronze-Silver-Gold-Serving medallion pattern in BigQuery, build robust ingestion from Cloud Storage and Cloud SQL, and set up datasets, partitions, and cost controls.

You will also implement batch and on-demand patterns, curate BI-ready datasets, and ensure data quality, lineage, and monitoring.

Qualifications

• Strong hands-on experience with Google Cloud data services.
• Strong BigQuery experience, including modeling, optimization, partitioning, clustering, and cost control.
• Experience with Cloud Storage ingestion patterns.
• Experience with Cloud SQL / MySQL to BigQuery integration.
• Experience building batch and incremental data pipelines.
• Experience with Dataform, dbt, Cloud Composer / Airflow, Dataflow, Cloud Run, or similar tools.
• Strong SQL and data modeling skills.
• Good understanding of lakehouse / medallion architecture.
• Experience with data quality, metadata, lineage, logging, and monitoring.
• Ability to work in an ambiguous consulting environment and translate high-level requirements into practical implementation plans.
• Strong communication skills and client-facing experience.

Responsibilities

• Assess the existing data pipeline, GCP architecture, data sources, and current limitations.
• Design a BigQuery-based medallion architecture: Bronze, Silver, Gold, and Serving layers.
• Design and/or implement data ingestion pipelines from Cloud Storage and Cloud SQL / MySQL into BigQuery.
• Define BigQuery datasets, tables, partitioning, clustering, and cost-control strategies.
• Support batch, incremental, and on-demand processing patterns.
• Design curated and BI-ready datasets for analytics and reporting.
• Support data quality checks, validation, logging, monitoring, and reprocessing patterns.
• Define integration patterns for Elasticsearch or other search / serving layers.
• Support architecture for AI enrichment, embeddings, semantic search, and media metadata where required.
• Prepare technical documentation, architecture diagrams, implementation roadmap, and handover materials.
• Work closely with client engineering, data, DevOps, and product teams.

Skills

GCP data services

BigQuery data modeling

Cloud Storage ingestion

Cloud SQL / MySQL to BigQuery

Batch and incremental pipelines

Dataform / dbt / Airflow

SQL / data modeling

lakehouse / medallion

data quality / lineage / monitoring

client-facing communication

Tools

Dataform

dbt

Cloud Composer / Airflow

Dataflow

Cloud Run

Job description

Category: IT Services Location:

Key Responsibilities

• Assess the existing data pipeline, GCP architecture, data sources, and current limitations.
• Design a BigQuery-based medallion architecture: Bronze, Silver, Gold, and Serving layers.
• Design and/or implement data ingestion pipelines from Cloud Storage and Cloud SQL / MySQL into BigQuery.
• Define BigQuery datasets, tables, partitioning, clustering, and cost-control strategies.
• Support batch, incremental, and on-demand processing patterns.
• Design curated and BI-ready datasets for analytics and reporting.
• Support data quality checks, validation, logging, monitoring, and reprocessing patterns.
• Define integration patterns for Elasticsearch or other search / serving layers.
• Support architecture for AI enrichment, embeddings, semantic search, and media metadata where required.
• Prepare technical documentation, architecture diagrams, implementation roadmap, and handover materials.
• Work closely with client engineering, data, DevOps, and product teams.

Requirements

• Strong hands-on experience with Google Cloud data services.
• Strong BigQuery experience, including modeling, optimization, partitioning, clustering, and cost control.
• Experience with Cloud Storage ingestion patterns.
• Experience with Cloud SQL / MySQL to BigQuery integration.
• Experience building batch and incremental data pipelines.
• Experience with Dataform, dbt, Cloud Composer / Airflow, Dataflow, Cloud Run, or similar tools.
• Strong SQL and data modeling skills.
• Good understanding of lakehouse / medallion architecture.
• Experience with data quality, metadata, lineage, logging, and monitoring.
• Ability to work in an ambiguous consulting environment and translate high-level requirements into practical implementation plans.
• Strong communication skills and client-facing experience.

Preferred Skills

• Experience with Elasticsearch or search index publishing.
• Experience with Power BI-ready datasets or analytical serving layers.
• Experience with Vertex AI, Gemini, embeddings, vector search, or semantic search.
• Experience with media, social media, marketing, campaign, influencer, or audience data.
• Experience with Dataplex, Data Catalog, IAM, policy tags, row-level security, or column-level security.
• Experience with CI/CD for data pipelines.

Benefits

• WFH or Hybrid and competitive salary Details

Originally posted on Himalayas