GCP Data Engineer (Google Cloud, BigQuery, PySpark & Cloud Composer) | Techedge Solution | Pan India

Techedge Solution; India; Remote; INR 1,500,000 - 2,700,000; Full time

Job summary

Techedge Solution is seeking an experienced GCP Data Engineer to lead end-to-end ELT pipelines, BigQuery optimization, and Dataproc PySpark processing across Pan India. The role demands strong cloud architecture skills and hands-on migration of on-prem Hadoop to GCP.

You will collaborate with cross-functional teams and deliver robust, scalable data platforms. The position offers permanent full-time engagement with competitive compensation and opportunities for professional development within a

Qualifications

• 5+ years of experience in cloud data engineering, big data analytics, and ETL/ELT development.
• Strong, demonstrable operational expertise in GCP data services and cloud architecture.
• Comprehensive working knowledge of BigQuery optimization, PySpark on Dataproc, and Cloud Composer (Airflow).
• Experience migrating on-prem Hadoop to cloud data platforms and implementing data quality frameworks.

Responsibilities

• Architect end-to-end ELT pipelines and cloud data architectures on GCP.
• Build and optimize BigQuery datasets with partitioning, clustering, and cost optimization.
• Execute large-scale PySpark on Dataproc data processing and transformations.
• Orchestrate workflows with Cloud Composer and Apache Airflow.
• Lead cloud modernization and migration of on-prem Hadoop to GCP.
• Implement data quality frameworks and performance tuning for high-volume datasets.
• Collaborate with data architects, BI analysts, DevOps, and stakeholders.
• Maintain documentation, data lineage maps, and runbooks.

Skills

BigQuery

PySpark

Dataproc

Cloud Composer

Python

SQL

Hadoop migration

GCP data services

ETL/ELT

Education

Bachelor’s degree in Computer Science, Data Science, Information Technology, Software Engineering, or related technical discipline

Tools

Jira

Confluence

GitHub

Terraform

Airflow

Looker

Job description

Position Summary:

Driving advanced Google Cloud Platform (GCP) data engineering, cloud modernization, and large-scale data migration across Pan India, Techedge Solution is actively seeking an experienced GCP Data Engineer for a permanent full-time professional engagement. In this vital technical role, you will take full operational, tactical, and technical ownership of end-to-end ELT pipeline development, BigQuery query optimization, Dataproc PySpark processing, and Cloud Composer DAG orchestration. Bringing 5+ years of progressive professional experience in cloud data engineering and big data analytics, you will empower an elite technology consulting enterprise to achieve uncompromised data pipeline throughput, secure cloud architectures, and superior enterprise analytics performance. This prestigious career opportunity offers competitive remuneration, professional development, and career growth within a leading international technical recruitment and IT consulting enterprise.

Detailed Job Description:

As a GCP Data Engineer supporting Techedge Solution on a professional basis across Pan India, you will take full operational, tactical, and technical ownership of cloud data platform construction, on-premises Hadoop to GCP migration projects, ETL/ELT workflow orchestration, and database performance tuning. Your day-to-day responsibilities include building and optimizing scalable data pipelines using Python, PySpark on Dataproc, and advanced SQL, managing BigQuery partitioning, clustering, and cost/query optimization, orchestrating complex workflows using Cloud Composer and Apache Airflow, implementing robust data transformation and data quality verification frameworks, collaborating with multidisciplinary data engineering squads during agile sprint cycles, and interacting directly with technical recruiters like Pavithran Chandran. Furthermore, you will manage application requirements by submitting your updated resume via direct email to Pavithran Chandran at [e-mail].

Key Responsibilities:

• Architect, design, develop, test, and maintain robust end-to-end ELT pipelines and cloud data architectures on Google Cloud Platform (GCP).
• Build, optimize, and manage high-performance BigQuery datasets utilizing advanced partitioning, clustering, and query cost optimization strategies.
• Execute large-scale distributed data processing and transformation pipelines utilizing PySpark on Google Cloud Dataproc.
• Orchestrate complex workflow Directed Acyclic Graphs (DAGs) and automated data pipelines utilizing Cloud Composer and Apache Airflow.
• Lead enterprise cloud modernization and migration initiatives, migrating legacy on-premises Hadoop clusters and relational databases to GCP.
• Implement rigorous data quality frameworks, data transformation protocols, and performance-tuning mechanisms for high-volume datasets.
• Collaborate seamlessly with data architects, business intelligence analysts, DevOps engineers, and client stakeholders.
• Maintain meticulous technical documentation, data lineage maps, pipeline runbooks, and cloud architecture compliance reports.

Required Qualifications & Skills:

• Bachelor’s degree in Computer Science, Data Science, Information Technology, Software Engineering, or related technical discipline.
• Minimum of 5+ years of progressive professional hands-on experience in cloud data engineering, big data analytics, and ETL/ELT development.
• Strong, demonstrable operational expertise in Google Cloud Platform (GCP) data services and cloud architecture.
• Comprehensive working knowledge and practical proficiency in BigQuery optimization, PySpark on Dataproc, and Cloud Composer (Airflow).
• Deep practical experience with Python programming, advanced SQL querying, and relational database management.
• Solid practical grasp of on-premises Hadoop to cloud data migration methodologies, data quality frameworks, and performance tuning.
• Exceptional analytical problem-solving aptitude, big data debugging capabilities, and professional stakeholder communication skills.
• Mandatory legal work authorization and physical availability for professional operations across Pan India.

Nice-to-Have Skills:

• Professional Google Cloud certifications such as Google Cloud Certified Professional Data Engineer or Google Cloud Certified Professional Cloud Architect.
• Prior professional working experience in top-tier technology consultancies, global IT services firms, or enterprise data product companies.
• Hands-on familiarity with additional GCP analytics tools (Dataflow, Pub/Sub, Datastream, Looker) and infrastructure as code (Terraform).
• Experience utilizing enterprise agile tracking and collaboration platforms (Jira, Confluence, GitHub).
• Active participation in data engineering user groups, Apache Airflow communities, or cloud computing forums.

Application Information:

• Salary/Rate: Market Competitive / Enterprise Cloud Data Package
• Work Schedule: Professional Engagement (Pan India)
• Deadline: Open until filled (Urgent Pan India GCP Data Engineer Opening)
• Notice Period: Immediate to 30 Days Preferred
• Contract Duration: Permanent / Full-Time Engagement (Pan India)

Recruitment Pro Tip:

When applying for this GCP Data Engineer position through Techedge Solution, ensure your resume explicitly highlights your BigQuery optimization tenure, PySpark/Dataproc distributed processing expertise, and your experience with Hadoop-to-GCP cloud migration projects.