Technical Lead – PySpark / Pune /

Tata Consultancy Services; Pune District; In loco; INR 2.500.000 - 3.200.000; Tempo pieno

Descrizione del lavoro

Tata Consultancy Services seeks a Technical Lead - PySpark to lead design and implementation of scalable data processing solutions. You will mentor the team, perform code reviews, and drive best practices across PySpark, Python, SQL, and Spark-based ETL pipelines.

Ideal candidates have hands-on PySpark expertise, strong Python skills, and experience optimizing Spark jobs, with a focus on scalable data engineering in a collaborative environment.

Competenze

• Strong hands-on development experience in PySpark.
• Strong programming skills in Python.
• Good understanding of Apache Spark and distributed data processing.
• Strong knowledge of SQL and data transformations.
• Experience in Spark/PySpark performance optimization.
• Good problem-solving and troubleshooting skills.
• Experience in technical leadership, code reviews, and team guidance.

Mansioni

• Design, develop, and maintain scalable data processing solutions using PySpark.
• Build and optimize ETL/data pipelines for large datasets.
• Perform performance tuning and optimization of Spark/PySpark jobs.
• Troubleshoot complex data processing and application issues.
• Conduct code reviews and ensure development best practices.
• Provide technical guidance and mentoring to team members.
• Collaborate with cross-functional teams to translate requirements into technical solutions.
• Ensure solutions are scalable, reliable, and maintainable.

Conoscenze

PySpark

Python

SQL

Apache Spark

data engineering

ETL/data pipelines

performance optimization

problem solving

technical leadership

code reviews

team mentoring

AWS/Azure/GCP

Git/CI-CD

big-data ecosystems

Strumenti

Git

CI/CD

Descrizione del lavoro

We are looking for a Technical Lead - PySpark with strong hands-on experience in building and optimizing scalable data processing solutions. The ideal candidate should have solid expertise in PySpark, Python, SQL, and data engineering, along with the ability to provide technical guidance to the development team.

Key Responsibilities

• Design, develop, and maintain scalable data processing solutions using PySpark.
• Build and optimize ETL/data pipelines for large datasets.
• Perform performance tuning and optimization of Spark/PySpark jobs.
• Troubleshoot complex data processing and application issues.
• Conduct code reviews and ensure development best practices.
• Provide technical guidance and mentoring to team members.
• Collaborate with cross-functional teams to translate requirements into technical solutions.
• Ensure solutions are scalable, reliable, and maintainable.

Required Skills

• Strong hands-on development experience in PySpark.
• Strong programming skills in Python.
• Good understanding of Apache Spark and distributed data processing.
• Strong knowledge of SQL and data transformations.
• Experience in Spark/PySpark performance optimization.
• Good problem-solving and troubleshooting skills.
• Experience in technical leadership, code reviews, and team guidance.

Preferred Skills

• Exposure to AWS, Azure, or GCP data platforms.
• Understanding of big-data and modern data-engineering ecosystems.
• Familiarity with Git, CI/CD, and software development best practices.