AWS Data Engineer + Pyspark

Tata Consultancy Services; Pune District; On-site; INR 900,000 - 1,300,000; Full time

Job summary

Tata Consultancy Services in Pune seeks an experienced data engineer to work on PySpark, Big Data, and AWS stack, delivering scalable data solutions and collaborating with business and IT stakeholders.

Ideal candidates bring 5+ years and strong SQL, Python, and Spark UI skills, with Agile experience and a focus on data architecture patterns like Data Vault and data mesh. On-site position with morning walk-ins available.

Qualifications

• 5+ years of total experience with PySpark, Big Data, AWS EMR, S3, IAM, Lambda, SNS, SQS, Python.
• Proficient in Spark UI/optimization and debugging techniques.
• Intermediate SQL exposure – Subqueries, Joins, CTEs; good DB technologies knowledge.
• Excellent communication to liaise with Business & IT stakeholders.
• Experience in Agile software development.

Responsibilities

• 5+ years of total experience with PySpark, Big Data, AWS EMR, S3, IAM, Lambda, SNS, SQS, Python.
• Spark UI/Optimization/debugging techniques
• Intermediate SQL exposure – Subquery, Joins, CTE’s
• Good understanding of Database technologies
• Excellent communication with Business & IT stakeholders
• Planning execution of a project and efforts estimation
• Understanding of Data Vault, data mesh and data fabric patterns
• Exposure to Agile ways of working

Skills

PySpark

Big Data

Python

SQL

Communication

Education

15 years of full-time education

Tools

IAM

Lambda

SNS

SQS

S3

EMR

Spark UI

Job description

Walkin Time: 9am-1pm

Experience Range: 5+ yrs

TCS has always been in the spotlight for being adept in the next big technologies. What we can offer you is a space to explore varied technologies and quench your techie soul.

Key Responsibilities

• 5+ years of total experience with good knowledge of Pyspark, Big Data experience, AWS EMR, S3, IAM, Lambda, SNS, SQS, Python and AWS services
• Spark UI/Optimization/debugging techniques
• Intermediate SQL exposure – Subquery, Joins, CTE’s
• Good understanding of Database technologies
• Excellent communication skills to liaise with Business & IT stakeholders.
• Expertise in planning execution of a project and efforts estimation.
• Understanding of Data Vault, data mesh and data fabric architecture patterns.
• Exposure to working in Agile ways of working.

Technical/Functional Skills

• IAM, Lambda, SNS, SQS
• Spark UI/Optimization/debugging techniques
• Good python scripting skills
• Intermediate SQL exposure – Subquery, Joins, CTE’s
• Database technologies
• AWS EMR, S3, IAM, Lambda, SNS, SQS

Minimum Qualification: 15 years of full-time education