Senior GCP BQ Data Engineer

Infosys; Hyderabad; On-site; INR 1,800,000 - 2,400,000; Full time

Job summary

Infosys in Hyderabad, India, is seeking an experienced Data Engineer to design, build, and maintain enterprise-scale data pipelines on Google Cloud Platform. You will implement ETL/ELT processes using Dataflow, Apache Beam, BigQuery and Cloud Composer to enable scalable analytics and reporting.

Responsibilities include designing data models, integrating sources via Pub/Sub and Cloud Storage, and optimizing performance. Strong Python/SQL skills and Git/CI-CD experience are required.

Qualifications

• 8 to 12 years of experience in Data Engineering, Big Data, and Cloud Data Platforms.
• Strong hands-on experience with Google Cloud Platform (GCP).
• Expertise in BigQuery for data warehousing, analytics, and performance optimization.
• Strong experience with Dataflow and Apache Beam for large-scale data processing.
• Experience with Apache Airflow and/or Cloud Composer for workflow orchestration.
• Hands-on experience with Cloud Spanner and distributed database concepts.
• Strong programming skills in Python and SQL.
• Experience building enterprise-grade ETL/ELT pipelines.
• Knowledge of Pub/Sub, Cloud Storage, Cloud Functions, and other GCP services.
• Strong understanding of data modeling, performance tuning, and data integration.
• Experience with version control systems such as Git.
• Knowledge of CI/CD, DevOps practices, and Agile methodologies.

Responsibilities

• Design, develop, and maintain enterprise-scale data engineering solutions on Google Cloud Platform (GCP).
• Build scalable ETL/ELT pipelines using Dataflow, Apache Beam, BigQuery, and Cloud Composer (Airflow).
• Design and optimize BigQuery data warehouses for large-scale analytics and reporting.
• Develop batch and real-time data pipelines using Dataflow and GCP-native services.
• Implement workflow orchestration using Apache Airflow / Cloud Composer.
• Design and manage data models and schemas in Cloud Spanner and BigQuery environments.
• Integrate multiple data sources using GCP services such as Pub/Sub, Cloud Storage, Cloud Functions, and Dataflow.
• Monitor, troubleshoot, and optimize pipeline performance, reliability, and scalability.
• Collaborate with architects, business stakeholders, and development teams to deliver cloud data

Skills

Python

SQL

Git

Agile methodologies

DevOps practices

Education

Engineering degrees including BTech/MBA/BSc/B.E.

Tools

Dataflow

Apache Beam

BigQuery

Cloud Composer (Airflow)

Airflow

Cloud Spanner

Pub/Sub

Cloud Storage

Cloud Functions

Job description

Educational Requirements

Bachelor of Engineering,BTech,BCA,BSc,ME,MTech,MCA,MSc,MBA

Service Line

Data Analytics Unit

Responsibilities

• Design, develop, and maintain enterprise-scale data engineering solutions on Google Cloud Platform (GCP).
• Build scalable ETL/ELT pipelines using Dataflow, Apache Beam, BigQuery, and Cloud Composer (Airflow).
• Design and optimize BigQuery data warehouses for large-scale analytics and reporting.
• Develop batch and real-time data pipelines using Dataflow and GCP-native services.
• Implement workflow orchestration using Apache Airflow / Cloud Composer.
• Design and manage data models and schemas in Cloud Spanner and BigQuery environments.
• Integrate multiple data sources using GCP services such as Pub/Sub, Cloud Storage, Cloud Functions, and Dataflow.
• Monitor, troubleshoot, and optimize pipeline performance, reliability, and scalability.
• Collaborate with architects, business stakeholders, and development teams to deliver cloud data

Technical and Professional Requirements

• 8 to 12 years of experience in Data Engineering, Big Data, and Cloud Data Platforms.
• Strong hands-on experience with Google Cloud Platform (GCP).
• Expertise in BigQuery for data warehousing, analytics, and performance optimization.
• Strong experience with Dataflow and Apache Beam for large-scale data processing.
• Experience with Apache Airflow and/or Cloud Composer for workflow orchestration.
• Hands-on experience with Cloud Spanner and distributed database concepts.
• Strong programming skills in Python and SQL.
• Experience building enterprise-grade ETL/ELT pipelines.
• Knowledge of Pub/Sub, Cloud Storage, Cloud Functions, and other GCP services.
• Strong understanding of data modeling, performance tuning, and data integration.
• Experience with version control systems such as Git.
• Knowledge of CI/CD, DevOps practices, and Agile methodologies.

Preferred Skills

• Technology->Big Data->Big Table->GCP
• Technology->Cloud Platform->GCP Core Services