Digital : Python(GCP, Pyspark)

Tata Consultancy Services; Hyderabad; On-site; INR 1,100,000 - 1,700,000; Full time

Job summary

Tata Consultancy Services in Hyderabad seeks a seasoned Big Data Engineer to design and implement data pipelines using Python, PySpark, and Spark. You will partner with data scientists and analytics teams to transform raw data into reliable, scalable insights.

The role demands strong knowledge of Hadoop, Hive, Kafka, SQL databases, and REST/JSON data formats, with exposure to GCP cloud services. You will optimize distributed computing workflows and ensure best practices in data engineering and

Qualifications

• 4 to 8+ years of hands-on experience with Python and PySpark.
• Strong Python programming proficiency and Spark ecosystem experience.
• Experience with Hadoop, Hive, Kafka and other Big Data technologies.
• Proficiency in SQL and databases (relational and NoSQL).
• Familiarity with REST APIs, JSON/XML, and data serialization.
• Exposure to GCP services and cloud computing environments.

Responsibilities

• Develop and optimize Python-based data pipelines using PySpark.
• Work with Spark, Hadoop, Hive, and Kafka for large-scale data processing.
• Collaborate on data modeling, data quality, and REST/JSON data integration.

Skills

Python

PySpark

Apache Spark

SQL

REST APIs

JSON/XML

Big Data

Hadoop

Kafka

Cloud (GCP)

Tools

Hive

Kafka

Cloud - GCP

Job description

Role & responsibilities

• 4 to 8 + Years of experience using Python and Pyspark.
• Strong proficiency in Python programming.
• Hands-on experience with PySpark and Apache Spark.
• Knowledge of Big Data technologies (Hadoop, Hive, Kafka, etc.).
• Experience with SQL and relational/non-relational databases.
• Familiarity with distributed computing and parallel processing.
• Understanding of data engineering best practices.
• Experience with REST APIs, JSON/XML, and data serialization.

Exposure to GCP services and cloud computing environments

Preferred candidate profile