Tcs Pune Walk-In-Drive GCP Data Engineer ( 5 To 12 yrs)

Tata Consultancy Services Limited; Pune District; On-site; INR 1,000,000 - 2,000,000; Full time

Job summary

Tata Consultancy Services Limited in Pune is seeking a Google Data Engineer to lead data platform modernization efforts. You will architect and implement data pipelines and data lakes on GCP, using BigQuery, Airflow, Dataflow, and related services.

You will collaborate with cross-functional teams to define KPIs, ensure scalable CI/CD for big data deployments, and ensure robust data delivery with Python and PySpark skills. English communication is essential.

Qualifications

• Bachelor's degree in CS or related field.
• Experience with Google Cloud big data tools (BigQuery, Airflow, Dataflow, Data Fusion).
• Proficient in Python and PySpark; strong Linux skills.
• Experience creating CI/CD pipelines for big data releases.

Responsibilities

• Data modelling/Data warehouse modernization/Cloud based data lakes.
• Create and maintain optimal data pipeline architecture.
• Define technical roadmap for data platform modernization and analysis and assessment of the existing data platforms.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Google cloud big data technologies.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Define KPIs for successful modernization of data platform and measure the improvement against the KPIs.

Skills

English communication

Team collaboration

Education

Bachelor's degree in CS or related

Tools

Google BigQuery

Airflow

Dataflow

Data Fusion

Cloud Storage

SQL

Python

PySpark

Linux

CI/CD pipelines

Job description

Role & responsibilities

Data modelling/Data warehouse modernization/Cloud based data lakes

Create and maintain optimal data pipeline architecture

Define technical roadmap for data platform modernization and analysis and assessment of the existing data platforms

Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Google cloud big data technologies.

Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

Define KPIs for successful modernization of data platform and measure the improvement against the KPIs.

Preferred candidate profile

Must-Have

• Google Data Engineer (Overall 5+ Years with 3 years relevant hands-on experience)
• Experience working in GCP based Big Data deployments (Batch/Realtime) leveraging components like GCP Big Query, air flow, Google Cloud Storage, Data fusion, Data flow, Data Proc etc.
• Good skills in Python Language, PYSPARK
• Good Skills in Linux
• Exposure to creation of CI-CD pipelines for promoting big data release deployments and designing log monitoring features.
• Excellent written and verbal communication skills in English
• The following skills are good-to-have, but not a requirement:

Good-to-Have

• Experience in On Premise Hadoop technologies (Hive, Sqoop, SPARK, Kafka)

Hadoop, Spark, Apache Beam