Lead GCP Data Engineer – ETL & BigQuery

Brillio; New York, Northern (NY, KY); Hybrid; USD 120,000 - 165,000; Full time

Job summary

Brillio is seeking an experienced GCP Data Engineer to design, develop, and optimize scalable data pipelines on Google Cloud Platform.

The ideal candidate has hands-on expertise in GCP Data Engineering, BigQuery, Dataflow, Dataproc, Python/PySpark and SQL, with solid knowledge of data warehousing and ETL processes, and a track record of delivering data-driven solutions for business stakeholders.

Qualifications

• Strong hands-on experience in GCP Data Engineering, BigQuery, Dataflow, Dataproc, Python/PySpark, and SQL, with solid knowledge of data warehousing and ETL processes.

Responsibilities

• Design and develop scalable ETL/ELT data pipelines on GCP.
• Build and optimize data solutions using BigQuery, Dataflow, Dataproc, Cloud Storage, and Pub/Sub.
• Develop data processing applications using Python, PySpark, and SQL.
• Perform data ingestion, transformation, cleansing, and validation.
• Design efficient data models and data warehouse solutions in BigQuery.
• Implement batch and real-time data processing pipelines.
• Optimize pipeline performance and BigQuery queries for scalability and cost efficiency.
• Integrate data from multiple sources into cloud-based data platforms.
• Implement data quality, security, monitoring, and governance practices.
• Work closely with Data Scientists, Architects, Analysts, and business stakeholders.
• Support cloud migration and modernization initiatives.

Skills

GCP Data Engineering

BigQuery

Dataflow

Dataproc

Python/PySpark

SQL

ETL processes

Tools

BigQuery

Dataflow

Dataproc

Cloud Storage

Pub/Sub

Python

PySpark

SQL

Job description

Brillio is seeking an experienced GCP Data Engineer to design, develop, and optimize scalable data pipelines on Google Cloud Platform.

The ideal candidate has hands-on expertise in GCP Data Engineering, BigQuery, Dataflow, Dataproc, Python/PySpark and SQL, with solid knowledge of data warehousing and ETL processes, and a track record of delivering data-driven solutions for business stakeholders.