Python,Pyspark,AWS

Tata Consultancy Services; Hyderabad; On-site; INR 1,700,000 - 2,600,000; Full time

Job summary

Tata Consultancy Services in Hyderabad seeks an experienced backend engineer to design and optimize data pipelines on AWS using S3, Lambda, and Step Functions. You will write Python code with PySpark for big data processing and ensure scalable, reliable performance.

You will collaborate with cloud-native architectures, monitor logs and KPIs, and contribute to robust, high-quality deployment practices in a fast-paced environment.

Qualifications

• 6-8 years of experience in backend development with expertise in AWS services (S3, Lambda).
• Strong proficiency in Python with hands-on PySpark for big data processing tasks.
• Experience building scalable architectures on cloud platforms like AWS; knowledge of containerization using Docker is an added advantage.

Responsibilities

• Design, develop, test, deploy, and maintain large-scale data processing pipelines using AWS services (S3, Lambda, Step Functions).
• Develop high-quality code in Python using PySpark for big data processing tasks.
• Ensure scalability, reliability, and performance by monitoring logs, metrics, and KPIs.

Skills

Python programming

PySpark

Backend development

Cloud architecture

Tools

AWS (S3, Lambda)

PySpark

Docker

Job description

Roles and Responsibilities

• Design, develop, test, deploy, and maintain large-scale data processing pipelines using AWS services such as S3, Lambda, Step Functions, etc.
  
• Develop high-quality code in Python using PySpark for big data processing tasks.
  
• Ensure scalability, reliability, and performance of deployed applications by monitoring logs, metrics, and other relevant KPIs.
  

Desired Candidate Profile

• 6-8 years of experience in backend development with expertise in AWS services (S3, Lambda).
  
• Strong proficiency in Python programming language with hands-on experience working with PySpark for big data processing tasks.
  
• Experience building scalable architectures on cloud platforms like AWS; knowledge of containerization using Docker is an added advantage.