Data Engineer (Python/Pyspark)

Tata Consultancy Services Limited; Kolkata District, Chennai, Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tata Consultancy Services Limited is seeking a Data Engineer (Python/PySpark) to design and implement scalable data pipelines and ETL/ELT workflows for large datasets. The role emphasizes Python and Spark expertise, SQL fluency, and cloud exposure across AWS/Azure/GCP.

You will collaborate with data architects and engineers to transform and validate data from multiple sources, optimize pipelines, and support analytics workloads with robust data solutions.

Qualifications

• Develop scalable data pipelines using Python and PySpark.
• Build and optimize ETL/ELT workflows for large datasets.
• Perform data transformation, cleansing, and validation.
• Write complex SQL queries and optimize data-processing workloads.
• Work with Spark-based distributed data-processing solutions.

Responsibilities

• Develop scalable data pipelines using Python and PySpark.
• Build and optimize ETL/ELT workflows for large datasets.
• Perform data transformation, cleansing, and validation.
• Write complex SQL queries and optimize data-processing workloads.
• Work with Spark-based distributed data-processing solutions.
• Integrate data from multiple structured and unstructured sources.
• Monitor and troubleshoot data pipelines and production issues.
• Collaborate with data architects, analysts, and engineering teams.

Skills

Python

PySpark

Apache Spark

SQL

ETL / Data Pipelines

Data transformation and processing

Cloud exposure: AWS / Azure / GCP

Git and CI/CD

Job description

Role- Data Engineer (Python/PySpark)

Desired Experience Range - 4-8 yrs

Location - Kolkata, Chennai, Bangalore

Mandatory Skills

• Python
• PySpark
• Apache Spark
• SQL
• ETL / Data Pipelines
• Data transformation and processing
• Cloud exposure: AWS / Azure / GCP
• Git and CI/CD

Roles & Responsibilities

• Develop scalable data pipelines using Python and PySpark.
• Build and optimize ETL/ELT workflows for large datasets.
• Perform data transformation, cleansing, and validation.
• Write complex SQL queries and optimize data-processing workloads.
• Work with Spark-based distributed data-processing solutions.
• Integrate data from multiple structured and unstructured sources.
• Monitor and troubleshoot data pipelines and production issues.
• Collaborate with data architects, analysts, and engineering teams.