Data Engineer – Data Platforms (GCP)

Classic Solutions; Haryana; On-site; INR 1,100,000 - 1,300,000; Full time

Job summary

Classic Solutions in India is seeking a Senior Data Engineer with strong Google Cloud Platform expertise to design, develop, and maintain scalable, cloud-native data solutions. The ideal candidate should have hands-on Python, SQL, PySpark and GCP data services, building enterprise-grade ETL/ELT pipelines for batch and streaming data.

You will design, optimize, and manage data workflows using BigQuery, Dataproc, Pub/Sub, and Cloud Composer, collaborating with stakeholders to deliver reliable data

Qualifications

• 5+ years of Data Engineering experience.
• Hands-on Python, SQL and PySpark.
• Strong GCP experience with BigQuery, Dataproc, Pub/Sub.
• Experience building production ETL/ELT pipelines.
• Knowledge of batch and streaming data processing.
• Data warehousing and modeling concepts.
• Git/GitHub and CI/CD practices.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Python and PySpark.
• Build cloud-native data solutions using GCP services.
• Develop and optimize SQL queries, transformations, and BigQuery workflows.
• Design and manage workflows with Cloud Composer (Airflow).
• Develop and optimize Spark apps on Dataproc.
• Build event-driven ingestion pipelines with Pub/Sub.
• Implement batch and real-time data processing for data warehouses/lakes.
• Ensure data quality with validation, monitoring, and error handling.
• Optimize pipeline performance and storage efficiency.
• Apply dimensional modeling, partitioning, and clustering concepts.

Skills

GCP

Python

SQL

PySpark

BigQuery

Airflow

Dataproc

Pub/Sub

Cloud Storage

ETL/ELT

Batch processing

Streaming

Data Warehousing

Data Modeling

Git

CI/CD

Tools

Git

Job description

Experience: 5-7 Years

Required Skills

Google Cloud Platform (GCP), Python, SQL, PySpark, BigQuery, Cloud Composer (Apache Airflow), Dataproc, Pub/Sub, Cloud Storage, ETL/ELT, Batch & Streaming Pipelines, Data Warehousing, Data Modeling, Git, CI/CD

Job Summary

We are seeking an experienced Senior Data Engineer with strong Google Cloud Platform (GCP) expertise to design, develop, and maintain scalable, cloud-native data solutions. The ideal candidate should have strong hands-on experience in Python, SQL, PySpark, and GCP data services, with expertise in building enterprise-grade ETL/ELT pipelines and batch and streaming data processing solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Python and PySpark.
• Build and optimize cloud-native data solutions using Google Cloud Platform services.
• Develop and optimize SQL queries, data transformations, and data processing workflows in BigQuery.
• Design and manage workflow orchestration using Cloud Composer (Apache Airflow).
• Develop, optimize, and troubleshoot Spark applications running on Dataproc.
• Build event-driven data ingestion pipelines using Pub/Sub.
• Implement batch and real-time data processing pipelines for data warehouses and data lakes.
• Ensure data quality through validation, reconciliation, monitoring, and exception handling.
• Optimize pipeline performance, data processing, and storage efficiency.
• Implement data warehouse concepts, including star and snowflake schemas, fact and dimension modeling, SCD, partitioning, and clustering.
• Follow engineering best practices using Git/GitHub, CI/CD, and code reviews.
• Collaborate with Product Owners, Business SMEs, and Engineering teams to deliver reliable data products.
• Create and maintain technical documentation for data pipelines and architecture.

Mandatory Skills

• 5+ years of experience in Data Engineering.
• Strong hands-on experience in Python, SQL, and PySpark.
• Hands-on experience with Google Cloud Platform (GCP).
• Experience with BigQuery, Dataproc, Pub/Sub, and Cloud Composer (Apache Airflow).
• Experience developing production-grade ETL/ELT pipelines.
• Knowledge of batch and streaming data processing.
• Experience with data warehouse concepts and data modeling.
• Understanding of Git/GitHub and CI/CD practices.

Preferred Skills

• Google Dataflow and Apache Beam.
• Bigtable, Cloud Spanner, Cloud SQL, and AlloyDB.
• Google Cloud Storage and Cloud Scheduler.
• dbt, Apache Spark, Hadoop, and Spark/Scala.
• Docker, data governance, and data quality practices.
• Experience with enterprise-scale cloud migration initiatives.
• Familiarity with AI-assisted development tools such as GitHub Copilot.

Ideal Candidate: A Senior Data Engineer with strong GCP data platform expertise, capable of building and optimizing scalable batch and streaming pipelines, enterprise ETL/ELT solutions, and cloud-native data architectures.

Pay: ₹1,100,000.00 - ₹1,300,000.00 per year

Work Location: In person