GCP Data Engineer

Tata Consultancy Services; Hyderabad; On-site; INR 800,000 - 1,200,000; Full time

Job summary

Tata Consultancy Services in Hyderabad is seeking a data engineer to create and maintain data pipelines, leveraging BigQuery, Pub/Sub, Dataflow, and Airflow to mirror existing processes.

You will analyze data migration needs, map datasets, ensure security with masking and encryption, and partner with data analytics experts to improve system functionality while embracing Agile and DevOps practices.

Qualifications

• Experience in BigQuery, Pub/Sub, Dataflow, Composer DAGs, and Airflow.
• Strong SQL knowledge with the ability to translate complex business scenarios into efficient queries.
• Experience in data modeling and data mapping.
• Hands-on experience with Google Cloud SDK and API scripting.
• Strong programming experience in Python.
• Experience working in Agile processes and DevOps methodologies.
• Retail domain experience will be an added advantage.

Responsibilities

• Create and maintain optimal data pipeline architecture, replicating existing processes.
• Analyze, explore, and select appropriate tools for data migration and transformation.
• Assemble large, complex datasets that meet functional and non-functional business requirements.
• Ensure data security through masking and encryption techniques.
• Work with data and analytics experts to improve data system functionality.
• Identify, design, and implement process improvements by automating manual processes, optimizing data delivery, and redesigning infrastructure for greater scalability.

Skills

SQL

Python

GCP BigQuery

Pub/Sub

Dataflow

Composer DAGs

Airflow

Data modeling

API scripting

Agile

DevOps

Retail domain experience

Tools

Google Cloud SDK

Job description

Virtual Interview: 10th Oct

Required Technical Skill Set:

SQL + GCP (BigQuery, Pub/Sub, Dataflow, Composer DAGs, Airflow)

Key Requirements:

• Experience in BigQuery, Pub/Sub, Dataflow, Composer DAGs, and Airflow.
• Strong SQL knowledge with the ability to translate complex business scenarios into efficient queries.
• Experience in data modeling and data mapping.
• Hands-on experience with Google Cloud SDK and API scripting.
• Strong programming experience in Python.
• Experience working in Agile processes and DevOps methodologies.
• Retail domain experience will be an added advantage.

Responsibilities:

• Create and maintain optimal data pipeline architecture, replicating existing processes.
• Analyze, explore, and select appropriate tools for data migration and transformation.
• Assemble large, complex datasets that meet functional and non-functional business requirements.
• Ensure data security through masking and encryption techniques.
• Work with data and analytics experts to improve data system functionality.
• Identify, design, and implement process improvements by automating manual processes, optimizing data delivery, and redesigning infrastructure for greater scalability.

We are not looking for immediate joiners. We are specifically seeking candidates who have a 90-day notice period.