GCP - Bigquery Developer

IDESLABS PRIVATE LIMITED; Pune District, Hyderabad; On-site; INR 8,00,000 - 12,00,000; Full time

Job summary

A leading tech company in Pune is looking for a skilled data engineer to develop and optimize ETL/ELT pipelines utilizing Google Cloud Platform. The ideal candidate will have proficiency in SQL, Python, and experience with BigQuery. Responsibilities include ensuring data quality and working independently to resolve technical issues. This role offers a chance to work on advanced data processing projects in a dynamic environment.

Qualifications

• Proficient in SQL and PL/SQL for data handling.
• Experience with Python for automation.
• Strong analytical skills for debugging data issues.

Responsibilities

• Develop and optimize ETL/ELT pipelines.
• Work extensively with BigQuery for data processing.
• Ensure data quality and integrity across pipelines.

Skills

SQL

PL/SQL

Python

Data modeling

Debugging skills

Tools

Google Cloud Platform

BigQuery

Cloud Storage

Cloud Logging

Dataproc

Pub/Sub

GitHub

Job description

Responsibilities

• Develop, implement, and optimize ETL/ELT pipelines for processing large datasets efficiently.
• Work extensively with BigQuery for data processing, querying, and optimization.
• Utilize Cloud Storage, Cloud Logging, Dataproc, and Pub/Sub for data ingestion, storage, and event-driven processing.
• Perform performance tuning and testing of the ELT platform to ensure high efficiency and scalability.
• Debug technical issues, perform root cause analysis, and provide solutions for production incidents.
• Ensure data quality, accuracy, and integrity across data pipelines.
• Collaborate with cross-functional teams to define technical requirements and deliver solutions.
• Work independently on assigned tasks while maintaining high levels of productivity and efficiency.

Qualifications / Skills

• Proficiency in SQL and PL/SQL for querying and manipulating data.
• Experience in Python for data processing and automation.
• Hands-on experience with Google Cloud Platform (GCP), particularly:
• • BigQuery (must-have)
  • Cloud Storage
  • Cloud Logging
  • Dataproc
  • Pub/Sub
• Experience with GitHub and CI/CD pipelines for automation and deployment.
• Performance tuning and performance testing of ELT processes.
• Strong analytical and debugging skills to resolve data and pipeline issues efficiently.
• Self-motivated and able to work independently as an individual contributor.
• Good understanding of data modeling, database design, and data warehousing concepts.