GCP Data Architect

Tredence; India; Hybrid; INR 3,500,000 - 7,000,000; Full time

Job summary

Tredence is seeking a seasoned GCP Data Architect to design and implement scalable data pipelines and analytics solutions across on-prem and cloud environments in India. You will lead data modeling, ETL/ELT processes, and governance, while mentoring junior engineers.

The role requires 8+ years of hands-on experience with GCP services (Airflow, Dataflow, BigQuery), Hadoop ecosystem, and streaming technologies. Location-based in multiple Indian cities, with strong collaboration across teams.

Qualifications

• Strong understanding of data warehousing and data modeling techniques.
• Proficient understanding of distributed computing principles - Hadoop v2, MapReduce, HDFS
• Strong data engineering skills on GCP cloud platforms Airflow, Cloud Composer, Data Fusion, Data Flow, Data Proc, Big Query
• Experience with building stream-processing systems, using solutions such as Storm or Spark- Streaming
• Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
• Experience with Spark, SQL, and Linux.
• Knowledge of various ETL techniques and frameworks, such as Flume, Apache NiFi, or DBT.
• Experience with various messaging systems, such as Kafka or RabbitMQ.
• Good understanding of Lambda Architecture, along with its advantages and drawbacks

Responsibilities

• Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
• Develop and maintain data pipelines implementing ETL process, monitoring performance and advising any necessary infrastructure changes
• Translate complex technical and functional requirements into detailed designs
• Investigate and analyze alternative solutions to data storing, processing etc. to ensure most streamlined approaches are implemented
• Serve as a mentor to junior staff by conducting technical training sessions and reviewing project outputs

Skills

Data modeling

ETL/ELT

GCP proficiency

Big data concepts

Mentoring

Tools

Airflow

Cloud Composer

Data Fusion

Dataflow

BigQuery

Storm

Spark Streaming

Pig

Hive

Impala

NiFi

DBT

Kafka

RabbitMQ

Job description

Role: GCP Data Architect

Exp: 8+ Yrs

Work Location: Bangalore, Hyderabad, Chennai, Pune, Gurugram, Kolkata

Primary Roles and Responsibilities:

• Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
• Develop and maintain data pipelines implementing ETL process, monitoring performance and advising any necessary infrastructure changes
• Translate complex technical and functional requirements into detailed designs
• Investigate and analyze alternative solutions to data storing, processing etc. to ensure most streamlined approaches are implemented
• Serve as a mentor to junior staff by conducting technical training sessions and reviewing project outputs

Skills and Qualifications:

• Strong understanding of data warehousing and data modeling techniques.
• Proficient understanding of distributed computing principles - Hadoop v2, MapReduce, HDFS
• Strong data engineering skills on GCP cloud platforms Airflow, Cloud Composer, Data Fusion, Data Flow, Data Proc, Big Query
• Experience with building stream-processing systems, using solutions such as Storm or Spark- Streaming
• Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
• Experience with Spark, SQL, and Linux.
• Knowledge of various ETL techniques and frameworks, such as Flume, Apache NiFi, or DBT.
• Experience with various messaging systems, such as Kafka or RabbitMQ.
• Good understanding of Lambda Architecture, along with its advantages and drawbacks