Pyspark Data Engineer

Tata Consultancy Services; Hyderabad; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Tata Consultancy Services in Bengaluru is inviting candidates for a Walk-in Drive for PySpark Data Engineer. The role targets professionals with 5-10 years of experience, skilled in PySpark, Python, SQL, and AWS cloud services.

The drive features hands-on sessions and enterprise-scale data projects to showcase strong data engineering capabilities. Key responsibilities include building scalable data pipelines, ETL processes, and performance tuning of Spark jobs, while collaborating with

Qualifications

• 5–10 years of relevant experience.
• Strong PySpark, Python and SQL skills.
• Experience with AWS services (EMR, S3, Lambda).
• Knowledge of Hadoop ecosystem and Hive/Impala.

Responsibilities

• Design, develop, and optimize scalable data pipelines using PySpark.
• Work on enterprise data engineering and analytics projects.
• Develop and maintain ETL processes for large datasets.
• Analyze Spark jobs with Spark UI and performance tuning.
• Write efficient SQL queries with joins, subqueries, and data transformations.
• Collaborate with cross-functional teams to deliver data solutions.
• Utilize AWS services (EMR, S3, Lambda, IAM, Redshift) for data workloads.
• Apply data quality, governance, and performance best practices.

Skills

Python Programming

PySpark

Big Data Technologies

Hadoop Ecosystem

Hive

Impala

SQL

Job description

Walk-in Drive | PySpark Data Engineer | Bangalore | 8th August

Are you passionate about Big Data, PySpark, and AWS technologies?
Join us for an exciting Walk-in Drive and be a part of a dynamic Data Engineering team working on cutting-edge cloud and big data solutions.

Position: PySpark Data Engineer

Location: Bangalore
Walk-in Date: 8th August
Experience Required: 5 - 10 Years

Required Technical Skills

• Python Programming
• PySpark
• Big Data Technologies
• Hadoop Ecosystem
• Hive, Impala
• SQL

Key Responsibilities

• Design, develop, and optimize scalable data pipelines using PySpark and Big Data technologies.
• Work on enterprise-level Data Engineering and Data Analytics projects.
• Develop and maintain ETL processes for large-scale datasets.
• Analyze and troubleshoot Spark jobs using Spark UI and performance tuning techniques.
• Write efficient SQL queries involving joins, subqueries, CTEs, and complex data transformations.
• Collaborate with cross-functional teams to understand business requirements and deliver data solutions.
• Work with AWS cloud services including EMR, S3, Lambda, SNS, SQS, IAM, and Redshift.
• Implement best practices for data quality, governance, and performance optimization.

Must-Have Skills

• Hadoop
• Spark / PySpark
• Python
• Hive
• Impala
• SQL
• Data Engineering Projects
• Unix/Linux Environment
• HDFS Commands