PySpark Developer with AWS

Tata Consultancy Services; Kolkata District, Hyderabad, Bengaluru; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Tata Consultancy Services in India is seeking a PySpark Developer with strong AWS expertise to design, develop, and optimize scalable data-processing solutions. You will work across modern data platforms and collaborate with cross-functional teams to deliver robust pipelines.

We expect hands-on experience with PySpark, Python, Spark, SQL, ETL/data pipelines, and AWS cloud services. The role involves tuning performance, ensuring data quality, and delivering reliable data products for enterprise

Qualifications

• Strong hands-on experience with PySpark and Spark.
• AWS cloud services experience required.
• Proficient in Python, SQL and ETL design.
• Experience building scalable ETL/data pipelines.
• Familiar with big data processing frameworks.
• Experience with AWS services such as S3, Glue, EMR, Lambda, Redshift, IAM and CloudWatch preferred.

Responsibilities

• Develop and maintain large-scale PySpark data processing applications.
• Build scalable ETL/ELT pipelines.
• Work with AWS services for data engineering workloads.
• Write and optimize complex SQL queries.
• Tune PySpark/Spark jobs for performance.
• Troubleshoot data quality and pipeline issues.
• Collaborate with data architects and engineering teams.

Skills

PySpark

AWS

Apache Spark

Python

SQL

ETL/Data Eng

Data Pipelines

Big Data Processing

Job description

PySpark Developer with AWS

Experience: 810 Years
Locations: Hyderabad / Bangalore / Kolkata
Primary Skill: PySpark
Mandatory Skill: AWS

Job Summary:
We are looking for an experienced PySpark Developer with strong AWS expertise to design, develop, and optimize scalable data-processing solutions. The candidate should have strong hands-on experience in PySpark, Python, Spark, SQL, ETL/data pipelines, and AWS cloud services.

AWS experience is mandatory for this position.

Mandatory Skills

• PySpark
• AWS – Mandatory
• Apache Spark
• Python
• SQL
• ETL / Data Engineering
• Data Pipelines
• Big Data Processing
• AWS services such as S3, Glue, EMR, Lambda, Redshift, IAM and CloudWatch

Key Responsibilities

• Develop and maintain large-scale data processing applications using PySpark.
• Build scalable ETL/ELT data pipelines.
• Work with AWS cloud services for data engineering workloads.
• Write and optimize complex SQL queries.
• Optimize PySpark/Spark jobs for performance and scalability.
• Troubleshoot data pipeline and data quality issues.
• Collaborate with data architects and engineering teams.