Lead Data Engineer - GCP

Blend; Hyderabad; On-site; INR 2,000,000 - 3,500,000; Full time

Job summary

Blend, a premier AI services provider, seeks a Senior GCP Data Engineer in Hyderabad, India. The role requires 6+ years of data engineering experience and strong hands-on work with GCP, Python, and SQL.

You will design scalable data pipelines, develop ETL/ELT processes, and collaborate with cross-functional teams to meet data requirements while ensuring data quality and governance. Excellent problem-solving and CI/CD practices are valued.

Qualifications

• 6+ years of experience in Data Engineering.
• Strong hands-on experience with Google Cloud Platform (GCP).
• Strong programming experience in Python.
• Strong expertise in SQL, including complex queries, joins, CTEs, window functions, and query optimization.
• Experience developing ETL/ELT data pipelines.
• Hands-on experience with one or more GCP services such as BigQuery, Cloud Storage (GCS), Cloud Composer / Airflow, Dataflow, Pub/Sub, Dataproc.

Responsibilities

• Design and develop scalable data pipelines using GCP services.
• Develop data processing and transformation solutions using Python and SQL.
• Build and optimize ETL/ELT pipelines for large-volume datasets.
• Work with business and technical teams to understand data requirements and deliver robust solutions.
• Perform data modeling, data validation, and performance optimization.
• Monitor and troubleshoot data pipelines and resolve production issues.
• Implement data quality, security, and governance best practices.
• Participate in code reviews and contribute to technical design and architecture discussions.

Skills

Data Engineering

Python

SQL

ETL/ELT pipelines

GCP

Tools

BigQuery

Cloud Storage

Cloud Composer / Airflow

Dataflow

Pub/Sub

Dataproc

Apache Spark / PySpark

Git / CI/CD

Terraform

GCP Dataform

Job description

Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. With a mission to fuel bold visions, Blend tackles significant challenges by seamlessly aligning human expertise with artificial intelligence. The company is dedicated to unlocking value and fostering innovation for its clients by harnessing world-class people and data-driven strategy. We believe that the power of people and AI can have a meaningful impact on your world, creating more fulfilling work and projects for our people and clients. For more information, visit www.blend360.com.

Job Description

We are looking for a Senior GCP Data Engineer with 6+ years of experience in designing, developing, and maintaining scalable data engineering solutions on Google Cloud Platform (GCP). The ideal candidate should have strong hands-on expertise in Python, SQL, data pipelines, and GCP data services.

Key Responsibilities

• Design and develop scalable and reliable data pipelines using GCP services.
• Develop data processing and transformation solutions using Python and SQL.
• Build and optimize ETL/ELT pipelines for large-volume datasets.
• Work with business and technical teams to understand data requirements and deliver robust solutions.
• Perform data modeling, data validation, and performance optimization.
• Monitor and troubleshoot data pipelines and resolve production issues.
• Implement data quality, security, and governance best practices.
• Participate in code reviews and contribute to technical design and architecture discussions.

Qualifications

• 6+ years of experience in Data Engineering.
• Strong hands-on experience with Google Cloud Platform (GCP).
• Strong programming experience in Python.
• Strong expertise in SQL, including complex queries, joins, CTEs, window functions, and query optimization.
• Experience developing ETL/ELT data pipelines.
• Hands-on experience with one or more GCP services such as:
• • BigQuery
  • Cloud Storage (GCS)
  • Cloud Composer / Airflow
  • Dataflow
  • Pub/Sub
  • Dataproc
• Experience with Apache Spark / PySpark is preferred.
• Good understanding of data warehousing, data modeling, and distributed data processing.
• Experience with Git and CI/CD practices.

Nice to Have

• Experience with Terraform or Infrastructure as Code.
• Exposure to GCP Dataform.
• Experience with real-time/streaming data pipelines.
• Knowledge of cloud security and IAM.
• Experience working in Agile environments.

Additional Information

All your information will be kept confidential according to EEO guidelines.