J_ Python ,Pyspark, GCP_ Virtual Interview

Tata Consultancy Services Limited; Hyderabad; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Tata Consultancy Services Limited is seeking a highly skilled data engineer with 6 to 8 years of experience in Hyderabad. The role centers on building scalable data pipelines using Python, PySpark, and Spark, with strong emphasis on ETL design, data integrity, and cloud integration.

Candidates should have a solid foundation in Big Data technologies and SQL, with exposure to REST APIs and data serialization.

Qualifications

• Strong Python programming skills and hands-on PySpark experience.
• Proficient in Spark ecosystem (Apache Spark) and Big Data technologies.
• Solid SQL skills with relational and non-relational databases.
• Experience with distributed computing and data engineering best practices.
• Familiarity with REST APIs, JSON/XML, and data serialization.
• Exposure to cloud environments (AWS/Azure/GCP).
• Nice to have data warehousing/data lakes or ML libraries.

Responsibilities

• Develop and maintain scalable data pipelines using Python and PySpark.
• Design and implement ETL processes.
• Optimize PySpark applications for performance and reliability.
• Collaborate with cross-functional teams to understand data requirements.
• Write clean, efficient code and perform code reviews.
• Ensure data integrity and quality across the data lifecycle.
• Integrate with cloud platforms (AWS/Azure/GCP) and manage large datasets.

Skills

Python

PySpark

Apache Spark

Big Data

SQL

Distributed computing

Data engineering

REST APIs

JSON/XML

Cloud computing

Tools

Hadoop

Hive

Kafka

Job description

Responsibility of / Expectations from the

Role :Python,Pyspark,GCP

6 to 8 Years Experience

Location : Hyderabad

Desired Competencies (Technical/Behavioral Competency)

Must-Have**

(Ideally should not be more than 3-5)

• Strong proficiency in Python programming.
• Hands-on experience with PySpark and Apache Spark.
• Knowledge of Big Data technologies (Hadoop, Hive, Kafka, etc.).
• Experience with SQL and relational/non-relational databases.
• Familiarity with distributed computing and parallel processing.
• Understanding data engineering best practices.
• Experience with REST APIs, JSON/XML, and data serialization.
• Exposure to cloud computing environments.

Good-to-Have

• 5+ years of experience in Python and PySpark development.
• Experience with data warehousing and data lakes.
• Knowledge of machine learning libraries (e.g., MLlib) is a plus.
• Strong problem-solving and debugging skills.
• Excellent communication and collaboration abilities.

Role & responsibilities

• Develop and maintain scalable data pipelines using Python and PySpark.
• Design and implement ETL (Extract, Transform, Load) processes.
• Optimize and troubleshoot existing PySpark applications for performance.
• Collaborate with cross-functional teams to understand data requirements.
• Write clean, efficient, and well‑
• Conduct code reviews and participate in design discussions.
• Ensure data integrity and quality across the data lifecycle.
• Integrate with cloud platforms like AWS, Azure, or GCP.
• Implement data storage solutions and manage large-scale datasets.Role & responsibilities