Hiring Big Data Developer - Python/Pyspark - Chennai/Bangalore

Tech Mahindra; Chennai District, Bengaluru; Hybrid; INR 900,000 - 1,400,000; Full time

Job summary

Tech Mahindra is seeking a Big Data Engineer to work on Python and Spark based data pipelines in Chennai/Bangalore. The role requires hands-on development, data manipulation using Pandas, and exposure to Spark core, Spark SQL, and Spark Streaming.

The candidate should be proficient in Python, Pandas, and have experience with Polars for efficient data processing, along with strong problem-solving skills. Collaboration with teammates and stakeholders is essential for success.

Qualifications

• Proficiency in Python programming.
• Experience with Pandas for data manipulation and analysis.
• Knowledge of Polars for efficient data processing.
• Strong problem-solving skills and attention to detail.

Responsibilities

• Lead and guide a team of junior Python developers.
• Write, test, and debug Python and Spark code throughout the development lifecycle.
• Design low-latency, high-availability data processing applications.
• Perform performance tuning, optimization, and automation to improve reliability.
• Collaborate with team members and stakeholders to deliver solutions.

Skills

Python

Spark

Hadoop

Hive

SQL

Unix Shell Scripting

Airflow

Control-M

Job description

Big Data Engineer

Primary Skills Python, Spark,Hadoop, Hive, SQL, Unix Shell Scripting, Airflow or Control M

Exp – 4- 12 Yrs

Location – Tech M Office – Chennai/Bangalore

Job Description

Through and hands on knowledge on Python and Spark Programming and writing efficient, reusable, and reliable Python code.

Well versed with:

• Very good proficiency in Python and Spark programming.
• Pandas: Experience with data manipulation and analysis using Pandas.
• Implementation experience of Spark Core, Spark SQL and Spark Streaming
• Working with Spark in combination Hadoop Ecosystem
• Design and implementation of low-latency, high-availability, and performance applications.
• Should lead and guide a team of junior Python developers.
• As Sr Python Developer responsibilities include coding, testing, debugging, and troubleshooting throughout the application development process.
• Performance tuning, improvement, balancing, usability, and automation.
• Collaborate with other team members and stakeholders.

Educational Qualification—

• Proficiency in Python programming.
• Experience with Pandas for data manipulation and analysis.
• Knowledge of Polars for efficient data processing.
• Strong problem-solving skills and attention to detail.

Ability to work collaboratively in a team environment