Gcp Data Engineer

V2soft; Chennai; Hybrid; INR 1,200,000 - 1,800,000; Full time

Job summary

V2Soft, Inc. in Chennai is seeking a data engineer to design and maintain scalable data pipelines on GCP, leveraging BigQuery, Dataflow, and Dataproc.

You will implement ETL/ELT workflows with Python, PySpark, SQL, and manage orchestration via Airflow/Cloud Composer, GCS and Pub/Sub, plus Terraform-based infrastructure automation.

Qualifications

• Minimum data engineering experience with cloud platforms.
• Hands-on expertise in GCP services and data tooling.
• Experience building scalable ETL/ELT pipelines and data modeling.

Responsibilities

• Design, develop and maintain scalable data pipelines on GCP.
• Build ETL/ELT workflows using BigQuery, Dataflow and Dataproc.
• Develop data processing solutions using Python, PySpark and SQL.

Skills

Python

PySpark

SQL

ETL/ELT pipelines

Tools

BigQuery

Dataflow

Dataproc

Airflow

Cloud Composer

Terraform

GCS

Pub/Sub

Cloud Run

Data Fusion

Cloud SQL

PostgreSQL

CI/CD tools

Job description

Role & responsibilities

• Design, develop and maintain scalable data pipelines on GCP.
• Build ETL/ELT workflows using BigQuery, Dataflow and Dataproc.
• Develop data processing solutions using Python, PySpark and SQL.
• Manage workflow orchestration using Apache Airflow / Cloud Composer.
• Work with GCS, Pub/Sub and other GCP services.
• Implement infrastructure automation using Terraform.
• Optimize BigQuery queries and data processing performance.
• Work on data integration, data modelling, validation and quality checks.
• Collaborate with cross-functional teams to deliver analytics and data products.

Preferred candidate profile

Candidates with 48 years of experience in Data Engineering and strong hands-on experience in Google Cloud Platform (GCP). Must have practical experience in BigQuery, Dataflow/Dataproc, Python, PySpark, SQL, Airflow/Cloud Composer and Terraform. Should have experience developing scalable ETL/ELT pipelines, handling large datasets, data integration and cloud-based data processing. Good knowledge of GCS, Pub/Sub and data modelling is expected. Candidates with experience in Cloud Run, Data Fusion, Cloud SQL, PostgreSQL and CI/CD tools will be preferred.