GCP Data Engineer

VMC Soft Technologies, Inc; Chennai District; On-site; INR 1,400,000 - 2,800,000; Full time

Job summary

VMC Soft Technologies, Inc. is seeking a GCP Data Engineer to design and manage data ingestion pipelines across GCS, BigQuery, Dataflow, Composer, Data Fusion, and DataProc for ETL/ELT operations.

Ideal candidates bring 5–12 years of experience, strong Python, Terraform, and Git skills, plus deep knowledge of BigQuery SQL optimization and data ingestion concepts. The role is based in multiple Indian locations with opportunities for growth.

Qualifications

• 5+ years of experience as a Data Engineer or in a similar role.
• Strong working knowledge of Google Cloud Storage (GCS), BigQuery, Composer, Data Fusion, Dataproc.
• Proficiency in Python for scripting, debugging, and automation tasks.
• Experience with Terraform for infrastructure-as-code.
• Knowledge of Git for version control.
• Understanding of data ingestion concepts, file formats, and ETL workflows.
• Ability to debug and resolve runtime data issues independently.
• Strong problem-solving and analytical skills.

Responsibilities

• Configure and build data ingestion pipelines using a GCP-based framework.
• Work with GCS, BigQuery, Dataflow, Composer, Data Fusion, DataProc for ETL/ELT operations.
• Lead data engineering/migration projects in GCP with Python and independent work.
• Manage parameters, job configurations, and metadata for ingestion.
• Debug and resolve data, parameter, and job execution issues.
• Escalate framework-related bugs to the core development team when needed.
• Monitor daily job runs and troubleshoot failures to meet SLAs.
• Collaborate with cross-functional teams for smooth delivery.

Skills

GCS

BigQuery

Composer

Data Fusion

DataProc

Python

Terraform

Git

Data ingestion concepts

ETL workflows

Tools

Git

Terraform

SQL

Job description

Job Description

Role

GCP Data Engineer

Required Technical Skill Set

Data Engineer with hands-on experience in Google Cloud Platform (GCP) services to work on implementing and managing data ingestion.

Work with services such as GCS, BigQuery, Dataflow, Composer, Data Fusion, DataProc for ETL/ELT operations.

No of Requirements

6

Desired Experience Range

5-12 Years

Location of Requirement

Pune, Hyderabad, Bangalore, Chennai, Gurgaon

Desired Competencies (Technical/Behavioral Competency)

Must-Have

• 5+ years of experience as a Data Engineer or in a similar role.
• Strong working knowledge of Google Cloud Storage (GCS), BigQuery, Composer, Data Fusion, Dataproc.
• Proficiency in Python for scripting, debugging, and automation tasks.
• Experience with Terraform for infrastructure-as-code.
• Knowledge of Git for version control.
• Understanding of data ingestion concepts, file formats, and ETL workflows.
• Ability to debug and resolve runtime data issues independently.
• Strong problem-solving and analytical skills.

Good-to-Have

• Exposure to TWSd or other enterprise job schedulers.
• Basic understanding of SQL optimization in BigQuery.

SN Responsibility of / Expectations from the Role

• Configure and execute data ingestion pipelines using our reusable GCP-based framework.
• Work with services such as GCS, BigQuery, Dataflow, Composer, Data Fusion, DataProc for ETL/ELT operations.
• Strong experience in data engineering/migration projects in GCP (along with Python) is required and should be able to work independently.
• Manage parameters, job configurations, and metadata for ingestion.
• Debug and resolve issues related to data, parameters, and job execution.
• Escalate framework-related bugs to the core development team when required.
• Monitor daily job runs, troubleshoot failures, and ensure SLAs are met.
• Collaborate with cross-functional teams for smooth delivery.
• Follow version control best practices using Git.
• Maintain deployment scripts and infrastructure configurations via Terraform.
• (Nice-to-have) Work with TWSd job scheduling/monitoring tools.
• Suggest improvements to the framework or processes based on usage experience.
• Contribute to small enhancements in the framework, where applicable.