Gcp Data Engineer

Cerebra; Hyderabad, Pune District, Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Cerebra is seeking a Data Engineer in Hyderabad to design, build, and maintain scalable data pipelines on Google Cloud Platform. You will work with Data Scientists, Analysts, and developers to deliver robust data solutions.

The role emphasizes BigQuery, Python, SQL, and ETL/ELT processes, with a focus on data quality, governance, and scalable data architectures in a cloud environment.

Qualifications

• GCP is mandatory and must be hands-on.
• BigQuery expertise is essential.
• Proficient in Python and SQL.
• Strong ETL/ELT and data pipeline development experience.
• Experience with cloud data engineering and data integration.
• Ability to build and optimize data warehouses and data lakes.
• Strong data modeling skills for structured and unstructured data.
• Experience with Apache Spark / PySpark.
• Proficiency with GCP data services such as Dataflow, Dataproc, Cloud Storage, Pub/Sub.

Responsibilities

• Design scalable data pipelines and ETL/ELT workflows.
• Develop ingestion, transformation, and integration processes.
• Work with large structured and unstructured datasets.
• Build and optimize cloud-based data warehouses and data lakes.
• Perform data cleansing, validation, and quality checks.
• Develop and optimize SQL queries and data processing jobs.
• Work with GCP data services and cloud-native technologies.
• Monitor, troubleshoot, and optimize production data pipelines.
• Collaborate with Data Scientists, Analysts, Developers, and business teams.

Skills

GCP

BigQuery

Python

SQL

ETL/ELT

Data Pipeline Development

Data Warehousing

Data Modeling

Apache Spark / PySpark

GCP Data Services (Dataflow, Dataproc,

Tools

Databricks

Apache Airflow / Cloud Composer

Kafka

Terraform

Git / CI-CD

Docker / Kubernetes

Data Lake / Lakehouse

Data Quality and Governance

Job description

Role & responsibilities

Position: Data Engineer

Job Description:

We are looking for a skilled Data Engineer with strong experience in Google Cloud Platform (GCP) to design, build, and maintain scalable data pipelines and data platforms.

Key Responsibilities:

• Design and develop scalable data pipelines and ETL/ELT workflows.
• Develop data ingestion, transformation, and integration processes.
• Work with large structured and unstructured datasets.
• Build and optimize cloud-based data warehouses and data lakes.
• Perform data cleansing, validation, and data quality checks.
• Develop and optimize SQL queries and data processing jobs.
• Work with GCP data services and cloud-native technologies.
• Monitor, troubleshoot, and optimize production data pipelines.
• Collaborate with Data Scientists, Data Analysts, Developers, and business teams.

Must-Have / Mandatory Skills:

• Google Cloud Platform (GCP) Mandatory
• BigQuery
• Python
• SQL
• ETL / ELT
• Data Pipeline Development
• Data Warehousing
• Data Modeling
• Apache Spark / PySpark
• GCP Data Services such as Dataflow, Dataproc, Cloud Storage, Pub/Sub
• Experience with cloud data engineering and data integration.

Good-to-Have Skills:

• Databricks
• Apache Airflow / Cloud Composer
• Kafka
• Terraform
• Git / CI-CD
• Docker / Kubernetes
• Data Lake / Lakehouse
• Data Quality and Governance