AWS Data engineer+ Pyspark

Tata Consultancy Services; Indore District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tata Consultancy Services is seeking an AWS Data Engineer + PySpark to design and implement scalable data pipelines in Pune. The role focuses on PySpark development with AWS services and data integration tools.

Ideal candidates have 5-8 years of experience and a strong background in ETL concepts, S3, Glue, EMR, Redshift, DMS and AppFlow, along with Qlik Replicate and data testing capabilities.

Qualifications

• PySpark experience in data engineering
• AWS ETL concepts, S3, Glue, EMR, Redshift, DMS, AppFlow
• Experience with Qlik Replicate
• Data testing and validation processes

Responsibilities

• Develop and optimize PySpark data pipelines on AWS stack
• Design and implement ETL processes using S3, Glue, EMR, Redshift, DMS and AppFlow
• Perform data testing, validation and quality checks
• Collaborate with data engineers and stakeholders to ensure reliable data delivery

Skills

PySpark

AWS

Qlik Replicate

Data Testing

Education

BCA

BSc CS

BEng

Tools

Hadoop

Teradata

CloudFormation

Terraform

Git

Job description

AWS Data Engineer + PySpark - Job Description AWS Data Engineer + PySpark

• Experience: 5-8 Years
• Location: Pune
• Job Function: Technology
• Role: Engineer

Key Responsibilities & Skills

• Must Have:
  • PySpark
  • AWS (ETL Concepts, S3, Glue, EMR, Redshift, DMS, AppFlow)
  • Qlik Replicate
  • Data Testing
• Nice To Have:
  • Hadoop
  • Teradata Background
  • IaC (CloudFormation / Terraform)
  • Git

Qualifications

• Bachelor of Computer Application
• Bachelor of Computer Science
• Bachelor of Engineering