Walk-in | Pyspark Data Engineer

Tata Consultancy Services; Chennai District; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Tata Consultancy Services in Chennai is seeking a PySpark Data Engineer to design, develop, and maintain scalable data processing solutions using PySpark and Spark. 4 to 15 years of relevant experience is expected, with a strong emphasis on building robust data pipelines and cloud-based data platforms.

The role requires expertise in PySpark, Spark SQL, Python, and real-time data ingestion using Spark Streaming and Kafka, with walk-in at Chennai on the specified date.

Qualifications

• 4+ years of hands-on experience with PySpark and Spark
• Proficient in Python and Spark SQL/DataFrames
• Experience building data pipelines and data transformation workflows
• Cloud experience with AWS, Azure, or Google Cloud
• Knowledge of real-time data ingestion using Spark Streaming and Kafka

Responsibilities

• Design, develop, and maintain data processing solutions using PySpark, Spark, and Python
• Develop and optimize data pipelines on cloud platforms
• Integrate data from relational, NoSQL sources and cloud storage
• Implement data transformation logic using PySpark and Python

Skills

PySpark

Apache Spark

Python

DataFrames & Spark SQL

Spark Streaming

Kafka

Cloud platforms

Data Processing concepts

Data Pipeline Development

Data Transformation

Data Integration

Tools

AWS

Azure

Google Cloud

PySpark

Apache Spark

Python

Job description

Role/Skill: Pyspark Data Engineer

Experience: 4 to 15 Years

Walkin Location: Chennai

Walkin Drive Date: 19-Sep-26, Saturday

JD:

Responsibilities:

• Data Processing: Design, develop, and maintain data processing solutions using PySpark, Apache Spark, and Python.
• Data Pipeline Development: Develop and optimize data pipelines using PySpark, Apache Spark, and cloud-based data platforms.
• Data Integration: Integrate data from various sources, including relational databases, NoSQL databases, and cloud storage.
• Data Transformation: Develop and implement data transformation logic using PySpark, Apache Spark, and Python.

• PySpark: knowledge of PySpark, Apache Spark, and Python.
• Data Processing: understanding of data processing concepts, including data ingestion, data transformation, and data storage.
• Cloud Experience: Experience with cloud-based data platforms, including AWS, Azure, or Google Cloud.
• Expertise in DataFrames & Spark SQL, Spark Streaming with Apache Kafka real-time data ingestion pipelines