Google Cloud Platform Developer

Cloudxtreme; Hyderabad, Pune District, Bengaluru; On-site; INR 150,000 - 230,000; Full time

Job summary

Cloudxtreme is seeking a data engineer to analyze data and build scalable data pipelines on GCP. The role involves writing advanced SQL queries in BigQuery, developing batch and streaming pipelines with Apache Beam on Dataflow, and orchestrating workflows with Airflow/Cloud Composer.

You will maintain code in Bitbucket, deploy with Terraform, and document outputs in Confluence. Collaboration with BSA for requirements is essential, plus Python basics.

Qualifications

• Strong SQL and ability to write complex queries for analytics.
• Experience with ETL concepts and data integration.
• Proficiency in Python and Java programming.
• Experience with Google Cloud data services (BigQuery, Dataflow, Pub/Sub).

Responsibilities

• Analyze data and write complex SQL queries in BigQuery for insights.
• Create batch and stream data pipelines using Java with Apache Beam on Dataflow.
• Orchestrate workflows using Airflow/Cloud Composer.
• Maintain code in Bitbucket and deploy with Terraform.
• Document outputs and update status in Jira and Confluence.
• Understand customer requirements led by BSA to deliver outputs.
• Basic knowledge in Python programming.

Skills

SQL

ETL concepts

Python

Java

Data analysis

Tools

BigQuery

Airflow

Cloud Composer

Dataflow

Apache Beam

Google Pub/Sub

Terraform

Python

Java

Bitbucket

Jira

Confluence

Job description

Role & responsibilities

• Analyze data and write complex SQL queries in GCP Big Query to extract required information.
• Expertise in ETL concepts
• Very strong SQL, ability to write complex SQL to perform common types of analysis and aggregations
• Create stream and batch data pipeline using Java in Apache beam with distributed data processing using Google Dataflow.
• Expert in Airflow, Cloud Composer, Dataflow, Cloud Storage balancing
• Understand customer requirement lead down by BSA and provide required data outputs.
• Orchestrate data pipeline jobs using Google cloud composer.
• Write code in Python / Java to receive streaming data from Google cloud pub/sub.
• Maintain code in given Bit bucket code repository and use Terraform and Python based data pipeline to deploy code in various cloud environment.
• Accept work, update status in JIRA and create documentation in Confluence.
• Basic knowledge in Python programming,