Gcp Data Engineer

IntraEdge Technology; Chennai District; Hybrid; INR 1,500,000 - 2,100,000; Full time

Benefits offered by this job

Hybrid work model

Job summary

IntraEdge Technology is hiring an experienced Data Engineer to design and optimize scalable data pipelines on Google Cloud Platform for our Chennai hybrid team. You will work on data lakes and warehouses to support analytics, ML, and BI initiatives.

The role requires 5+ years in data engineering with strong GCP expertise, SQL, Python, and PySpark. You will collaborate with data scientists and business stakeholders to deliver high-quality data solutions.

Qualifications

• 5+ years of experience in data engineering with at least 3+ years on GCP.
• Strong SQL, Python, PySpark and/or Java/Scala for data processing.
• Experience with GCP services like BigQuery, Dataflow, Pub/Sub and Airflow.

Responsibilities

• Design and implement scalable data pipelines using GCP-native services (BigQuery, Dataflow, Airflow).
• Develop ETL/ELT processes to ingest data from APIs, files, databases, and streaming sources.
• Build and maintain data lakes and data warehouses on GCP with optimized schemas and partitioning.
• Create and manage batch and real-time workflows with Cloud Composer and Dataflow.
• Collaborate with data scientists and stakeholders to enable analytics and ML initiatives.

Skills

SQL

Python

PySpark

Java/Scala

Data Modeling

Education

Bachelor's or Master's degree in CS/Engineering/IS

Tools

GCP

BigQuery

Dataflow

Pub/Sub

Cloud Composer (Airflow)

Job description

IntraEdge is hiring experienced Data Engineers with deep expertise in Google Cloud Platform (GCP) to join our Data Engineering team (current client/project- American Express) for Chennai location (Hybrid - Weekly 3 days in office). In this role, you will design, build, and optimize scalable data pipelines, data lakes, and data warehouses on GCP to support advanced analytics, machine learning, and business intelligence initiatives across the organization. This is a high-impact role for someone who thrives in designing end-to-end data architectures, optimizing performance, and enabling data-driven decision-making at scale.

Key Responsibilities
Data Engineering & Architecture

• Design and implement scalable and secure data pipelines using GCP-native services (e.g., BigQuery, Dataflow, Airflow).
• Develop ETL/ELT processes to ingest data from diverse sources including APIs, flat files, databases, and streaming sources.
• Build and maintain data lakes and data warehouses on GCP, ensuring schema optimization, partitioning, and cost-efficiency.
• Create and manage batch and real-time data workflows using Cloud Composer (Apache Airflow) and Dataflow (Apache Beam).

Data Management & Governance

• Ensure data quality, integrity, and reliability through robust validation and monitoring frameworks.
• Implement best practices for data lineage, metadata management, and data cataloging using tools like Data Catalog and Looker.
• Collaborate with data governance, security, and compliance teams to enforce data access controls and encryption.

Collaboration & Leadership

• Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality solutions.
• Provide technical mentorship and code reviews for junior data engineers.
• Contribute to architecture reviews and technology evaluations for continuous improvement of the data platform.

Minimum Qualifications

• Bachelor's or Masters degree in Computer Science, Engineering, Information Systems, or a related field.
• 5+ years of experience in data engineering, with at least 3+ years working extensively on GCP.
• Strong proficiency with SQL, Python, PySpark and/or Java/Scala for data processing and scripting.
• Proven experience with GCP services such as:
• BigQuery
• Dataflow
• Pub/Sub
• Cloud Composer (Airflow)
• Experience in building large-scale ETL/ELT pipelines and optimizing query performance on cloud platforms.
• Solid understanding of data modeling, partitioning, and schema design for analytical workloads.
• Familiarity with CI/CD practices, Terraform, and version control (e.g., Git).