Pyspark Data Engineer

Tata Consultancy Services Limited; Pune District; Vor Ort; INR 1.000.000 - 1.400.000; Vollzeit

Zusammenfassung

Tata Consultancy Services Limited in Pune is seeking a PySpark Specialist with 5+ years of experience in PySpark and Big Data. The role involves building and optimizing data pipelines, leveraging AWS services, and collaborating with business and IT stakeholders.

Candidates should have strong SQL skills (Subqueries, Joins, CTEs) and knowledge of data architecture patterns such as Data Vault and Data Mesh. Agile experience is preferred and communication skills are essential.

Qualifikationen

• 5+ years of total experience with PySpark and Big Data.
• Proficiency in PySpark, AWS EMR, S3, IAM, Lambda, and other AWS services.
• Experience with Spark UI, optimization and debugging techniques.
• Intermediate SQL exposure – Subquery, Joins, CTEs.
• Understanding of database technologies and data architecture patterns.
• Excellent communication with Business & IT stakeholders.
• Experience in Agile projects and estimation.
• Knowledge of Data Vault, data mesh and data fabric patterns.
• Agile ways of working.

Kenntnisse

Pyspark

Big Data

Python

AWS EMR

S3

IAM

Lambda

SQL (Subqueries/Joins/CTEs)

Spark UI

Communication skills

Jobbeschreibung

Greetings from TATA CONSULTANCY SERVICES!

TCS is hiring for Pyspark

Location: - Pune

Experience Range: - 5+ years

• 4 – 7 years of total experience with good knowledge of Knowledge of Pyspark, Big Data experience, AWS EMR, S3, IAM, Lambda, SNS, SQS, Python and AWS services
• Spark UI/Optimization/debugging techniques
• Intermediate SQL exposure – Subquery, Joins, CTE’s
• Good understanding of Database technologies
• Excellent communication skills to liaise with Business & IT stakeholders.
• Expertise in planning execution of a project and efforts estimation.
• Understanding of Data Vault, data mesh and data fabric architecture patterns.
• Exposure to working in Agile ways of working.