GCP Data Engineer

Persistent Systems; Bengaluru; Hybrid; INR 1,500,000 - 2,500,000; Full time

Benefits offered by this job

Talent development programs

Higher education sponsorship

Flexible work hours

Long Service awards

Insurance coverage: life, personal/acc

Job summary

Persistent Systems in Bengaluru seeks an experienced GCP Data Engineer to support a large-scale migration from on-prem Teradata to Google BigQuery. You will design scalable data pipelines, optimize data models, enabling secure and compliant data migration within the GCP ecosystem.

You will lead ETL/ELT workflows with Cloud Composer, Pub/Sub, and Cloud Functions, building data warehouses and data marts while collaborating with analytics, reporting, and regulatory teams.

Qualifications

• 5 to 10 years of experience in Data Engineering, Data Warehousing, or Cloud Platform Engineering.
• Minimum 3+ years of hands-on experience with Google Cloud Platform (GCP).
• Strong hands-on programming experience in Python.
• Strong SQL expertise with experience in complex query development and optimization.
• Experience building and supporting data pipelines using Cloud Composer (Airflow).
• Experience implementing event-driven architectures using Pub/Sub and Cloud Functions.

Responsibilities

• Design and develop batch and event-driven data pipelines on Google Cloud Platform.
• Build scalable enterprise data warehouses and data marts using BigQuery.
• Lead migration activities from Teradata to BigQuery, including schema conversion, SQL transformation, and validation.
• Develop and maintain workflow orchestration solutions using Cloud Composer (Apache Airflow).
• Implement event-driven architectures using Pub/Sub and Cloud Functions.
• Develop data ingestion frameworks to load data from multiple enterprise source systems into Cloud Storage and BigQuery.
• Optimize BigQuery workloads through partitioning, clustering, materialized views, and query tuning.
• Design and implement data models that support analytics, reporting, and regulatory requirements.

Skills

Python

SQL

BigQuery

Airflow

GCP

Data pipelines

Data modeling

IAM security

Stakeholder communication

Education

Bachelor's Degree in Computer Science / IT / Engineering / MCA / BCA

Tools

Cloud Composer (Airflow)

Pub/Sub

Cloud Functions

Cloud Storage

IAM

Job description

We are seeking an experienced GCP Data Engineer to support the migration of a large-scale wholesale banking data mart from on-premises Teradata to Google BigQuery. The ideal candidate will possess strong expertise in Google Cloud Platform, BigQuery, Python, and cloud-native data engineering practices. You will be responsible for designing scalable data pipelines, optimizing data models, enabling secure and compliant data migration, and delivering high-performance analytics solutions within the GCP ecosystem.

• Experience: 5 to 10 Years
• Job Type: Full-Time Employment

What You’ll Do:

• Design and develop batch and event-driven data pipelines on Google Cloud Platform.
• Build scalable enterprise data warehouses and data marts using BigQuery.
• Lead migration activities from Teradata to BigQuery, including schema conversion, SQL transformation, and validation.
• Develop and maintain workflow orchestration solutions using Cloud Composer (Apache Airflow).
• Implement event-driven architectures using Pub/Sub and Cloud Functions.
• Develop data ingestion frameworks to load data from multiple enterprise source systems into Cloud Storage and BigQuery.
• Optimize BigQuery workloads through partitioning, clustering, materialized views, and query tuning.
• Design and implement data models that support analytics, reporting, and regulatory requirements.
• Develop Python-based utilities and frameworks for automation, data transformation, monitoring, and reconciliation.
• Manage and secure GCP resources using IAM roles, service accounts, and security best practices.
• Implement operational monitoring, logging, alerting, and troubleshooting processes.
• Collaborate with analytics, reporting, business, and technology teams to understand data requirements and deliver solutions.
• Ensure compliance with enterprise security, governance, audit, and regulatory standards.
• Support production deployments, post-migration stabilization, and platform optimization initiatives.

Expertise You’ll Bring:

• Bachelor's Degree in Computer Science, Information Technology, Engineering, MCA, BCA, or related discipline.
• 5 to 10 years of experience in Data Engineering, Data Warehousing, or Cloud Platform Engineering.
• Minimum 3+ years of hands‑on experience working with Google Cloud Platform (GCP).
• Strong hands‑on programming experience in Python.
• Strong SQL expertise with experience in complex query development and optimization.
• Experience building and supporting data pipelines using Cloud Composer (Apache Airflow).
• Experience implementing event‑driven architectures using Pub/Sub and Cloud Functions.
• Hands‑on experience with Cloud Storage, IAM, Service Accounts, and other core GCP services.
• Experience in data migration projects involving Teradata, on‑premises platforms, or legacy data warehouses.
• Strong knowledge of data warehousing, dimensional modeling, ETL/ELT frameworks, and data integration concepts.
• Experience implementing cost governance, chargeback reporting, and cloud operational monitoring solutions.
• Understanding of data lineage, metadata management, and data quality frameworks.
• Experience working with Banking, Financial Services, Risk Management, or Regulatory Reporting environments.
• Strong analytical, troubleshooting, and problem‑solving capabilities.
• Experience working within Agile delivery environments.
• Excellent communication and stakeholder management skills.
• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting‑edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

Let’s unleash your full potential at Persistent - persistent.com/careers

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."