Gcp Data Engineer

Datametica; Hyderabad; On-site; INR 900,000 - 1,500,000; Full time

Benefits offered by this job

Challenging role in fast-paced env

Work with cutting-edge GCP tech

Collaborative team culture

Continuous learning and growth

Job summary

Datametica is seeking a Senior Data Engineer in Hyderabad to design and implement scalable ETL pipelines on Google Cloud Platform. You’ll work with BigQuery, Cloud Composer, and Cloud Storage, building robust data solutions and optimizing performance and cost.

The ideal candidate has strong SQL/PL/SQL and Python skills, hands-on Airflow DAG development, and experience with data quality and automation in an agile environment. Collaboration with data scientists and engineers is essential.

Qualifications

• Must have hands-on ETL design and implementation experience.
• Strong SQL/PL-SQL skills with complex transformations.
• GCP skills: BigQuery, Cloud Composer, Cloud Storage.

Responsibilities

• Design, develop, and deploy scalable ETL pipelines using GCP data services (BigQuery, Cloud Composer, Cloud Storage).
• Develop and manage complex DAGs in Apache Airflow for data workflows.
• Write and optimize SQL/PL/SQL queries, procedures and functions for data manipulation.
• Optimize BigQuery workloads for performance and cost.
• Develop Python and Shell scripts for automation and data movement.
• Ensure data quality, integrity, and reliability across pipelines.
• Collaborate with data scientists, analysts, and engineers to meet data requirements.
• Participate in code reviews and establish data engineering best practices.
• Troubleshoot and resolve data pipeline issues promptly.
• Use Git for version control and collaboration.
• Stay updated on data engineering trends and cloud technologies.

Skills

ETL pipeline design

SQL

PL/SQL

GCP

BigQuery

Cloud Composer

Airflow

Python

Shell scripting

Git

Education

Bachelor’s degree in Computer Science or Engineering

Tools

Informatica

Job description

Job Title: Data Engineer
Location: Hyderabad
Experience Level: 3-8 Years

About the Role:

We are seeking a highly skilled and experienced Senior Data Engineer to join our growing team in Pune. The ideal candidate will have a strong background in data engineering, with a particular focus on Google Cloud Platform (GCP), Apache Airflow, and end-to-end ETL pipeline development. You will be responsible for designing, developing, and maintaining robust and scalable data pipelines, ensuring data quality, and optimizing data solutions for performance and cost. This role requires a hands-on approach and the ability to work both independently and collaboratively in an agile environment.

Responsibilities:

• Design, develop, and deploy scalable ETL pipelines using GCP data services, including BigQuery, Cloud Composer (Apache Airflow), and Cloud Storage.
• Develop, deploy, and manage complex DAGs in Apache Airflow for orchestrating data workflows.
• Write and optimize complex SQL and PL/SQL queries, stored procedures, and functions for data manipulation, transformation, and analysis.
• Optimize BigQuery workloads for performance, cost efficiency, and scalability.
• Develop scripts using Python and Shell scripting to support automation, data movement, and transformations.
• Ensure data quality, integrity, and reliability across all data solutions.
• Collaborate with cross-functional teams including data scientists, analysts, and engineers to understand data requirements and deliver effective solutions.
• Participate in code reviews and contribute to establishing and maintaining data engineering best practices.
• Troubleshoot and resolve data pipeline issues in a timely manner.
• Use version control systems (e.g., Git) for managing code and collaborating on engineering work.
• Stay updated on the latest trends and technologies in data engineering, cloud computing, and ETL processes.

Required Skills and Qualifications:

• 38 years of hands-on experience in Data Engineering roles.
• Mandatory Skills:
  • ETL pipeline design and implementation
  • SQL and PL/SQL (complex queries, procedures, and transformations)
  • Google Cloud Platform (GCP) — BigQuery, Cloud Composer, Cloud Storage
  • Apache Airflow (designing and deploying complex DAGs)
  • Python for scripting and data processing
  • Shell Scripting for automation and orchestration tasks
• Experience with Informatica is a strong plus.
• Proven ability to optimize BigQuery for performance and cost.
• Familiarity with Git and version control best practices.
• Excellent problem-solving, analytical, and communication skills.
• Ability to thrive both independently and within a collaborative agile team.
• Bachelor’s degree in Computer Science, Engineering, or a related field.

What We Offer:

• A challenging and rewarding role in a dynamic and fast-paced environment.
• Opportunity to work with cutting-edge technologies on the Google Cloud Platform.
• A collaborative and supportive team culture.
• Continuous learning and professional growth opportunities.