Senior GCP Data Engineer: Real-Time & BigQuery Expert

CoreAi Consulting

Phoenix (AZ)

On-site

USD 130,000 - 180,000

Full time

2 days ago

Be an early applicant

Application generator

A complete application in a minute — tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

CoreAi Consulting seeks a seasoned GCP Data Engineer to design and operate scalable data pipelines across ingestion, transformation, and serving layers. You will implement batch and real-time streaming using PySpark, Dataflow, Pub/Sub, and BigQuery, building reusable data frameworks and secure, observable data platforms.

The role emphasizes hands-on engineering, cost-aware architecture, and collaboration with stakeholders to deliver robust data products.

Qualifications

5+ years in data engineering with production pipelines.
Strong Python programming and Spark SQL expertise.
Deep knowledge of GCP data services (BigQuery, Dataflow, Pub/Sub).
Experience with batch and streaming pipelines.
Experience with Airflow DAGs and Cloud Composer.
Familiarity with data modeling (star/snowflake).
Experience with CI/CD and testing practices.

Responsibilities

Design, develop, and maintain enterprise-scale batch and streaming data pipelines using Python, PySpark, Dataflow, Dataproc, Pub/Sub, BigQuery, and Cloud Composer.
Build modular, reusable Python data ingestion and transformation frameworks.
Develop high-performance data-processing solutions using PySpark, Spark SQL, and Apache Beam.
Create and maintain workflow orchestration with Cloud Composer and Airflow.