GCP Data Engineer

ZettaMine Labs Pvt. Ltd.; Hyderabad; On-site; INR 1,800,000 - 3,200,000; Full time

Job summary

ZettaMine Labs Pvt. Ltd. in Hyderabad, India seeks a skilled Data Engineer to design scalable data pipelines on GCP, implement BigQuery data models, and optimize data flows for analytics.

You will build ETL/ELT processes, migrate legacy data to the cloud, and ensure data governance and security across platforms. You will collaborate with analysts and data scientists, leveraging Python and modern BI tools to deliver data-driven solutions while reducing cost through query optimization and

Qualifications

• Hands-on with Google Cloud Platform (GCP) including BigQuery and storage services.
• Expertise in BigQuery SQL: complex queries, optimization, partitioning and views.
• Python for data engineering and automation; ETL/ELT pipelines end-to-end.
• Experience with Airflow / Cloud Composer and data warehousing concepts.

Responsibilities

• Design and build scalable data pipelines on GCP to support analytics.
• Develop and optimize BigQuery data models, datasets, tables and views.
• Create ETL/ELT workflows for batch and real-time processing.
• Integrate data from multiple sources into enterprise data warehouses.
• Tune BigQuery queries for performance and cost efficiency.
• Maintain orchestration workflows using Airflow / Cloud Composer.
• Support cloud migrations from legacy platforms to GCP.
• Ensure data quality, governance, and security across platforms.
• Collaborate with stakeholders to deliver data-driven solutions.

Skills

GCP Expertise

BigQuery SQL

ETL/ELT pipelines

Python for data engineering

Airflow / Cloud Composer

Data modeling

Data warehousing

CI/CD & Git

Tools

Pub/Sub

Dataproc

Dataflow

Cloud Functions

Cloud Storage

Terraform

Looker/Tableau/Power BI

Job description

• Design and develop scalable data pipelines using Google Cloud Platform (GCP) services.
• Build and optimize data models, datasets, tables, views, and stored procedures in BigQuery.
• Develop ETL/ELT workflows for batch and real-time data processing.
• Integrate data from multiple sources into enterprise data warehouses.
• Perform BigQuery SQL performance tuning, query optimization, and cost optimization.
• Develop and maintain orchestration workflows using Airflow / Cloud Composer.
• Support cloud migration initiatives from legacy platforms such as Hadoop, Teradata, Oracle, or DataStage to GCP.
• Ensure data quality, governance, reliability, and security across data platforms.
• Collaborate with business stakeholders, analysts, and data scientists to deliver data-driven solutions.

Mandatory Skills

• Strong hands-on experience with Google Cloud Platform (GCP).
• Expertise in BigQuery SQL, including:
• Complex SQL development
• Query optimization
• Partitioning and Clustering
• Stored Procedures
• Views and Materialized Views
• Experience with BigQuery Data Warehouse development.
• Hands-on experience in Python for data engineering and automation.
• Experience with Cloud Storage, Pub/Sub, Dataproc, Dataflow, and Cloud Functions.
• Experience building and maintaining ETL/ELT pipelines.
• Experience with Apache Airflow / Cloud Composer.
• Knowledge of data modeling and data warehousing concepts.
• Experience with Git and CI/CD practices.

Good to Have

• PySpark and Spark.
• DBT (Data Build Tool).
• Terraform and Infrastructure as Code.
• Looker, Tableau, or Power BI.
• Exposure to real-time streaming solutions.
• Experience in cloud migration projects involving Teradata, Hadoop, Oracle, or Snowflake to GCP.

Preferred Experience

• Banking, Financial Services & Insurance (BFSI), Retail, or Analytics domains.
• Expertise in performance tuning and cost optimization of BigQuery workloads.
• Exposure to modern Data Lake and Data Warehouse architectures.