Lead Bigdata Engineer - Chennai/ Bangalore Location

Tech Mahindra; Chennai District; Hybrid; INR 2,400,000 - 4,200,000; Full time

Job summary

Tech Mahindra is seeking a Data Engineering Hadoop Developer to design, build, and optimize scalable data pipelines within the Hadoop ecosystem. The role focuses on batch/stream processing, data integration, and performance tuning, requiring strong SQL and data modeling skills.

The candidate will operate in Chennai/Bangalore with a hybrid model (3 days on-site per week), collaborating with business teams to deliver robust data solutions and governance.

Qualifications

• 8 years of hands-on experience in Hadoop/big data development.
• Strong knowledge of HDFS, Hive, Spark.
• Strong Python/Scala programming skills.

Responsibilities

• Design and develop robust ETL/data pipelines using Hadoop ecosystem tools.
• Build and optimize large-scale data processing jobs using Spark and/or Hive.
• Develop workflows using Airflow.
• Write complex SQL/HiveQL for data transformation and reporting needs.
• Ingest structured and unstructured data from multiple sources (Kafka, Sqoop, APIs, files).
• Ensure data quality, lineage, and governance standards are followed.
• Monitor, troubleshoot, and tune jobs for performance and scalability.
• Create technical documentation and follow coding best practices.

Skills

Hadoop/big data development

HDFS

Hive

Spark

Python/Scala

SQL

Airflow

Linux shell scripting

Git

Agile delivery

Tools

Sqoop

Git

Job description

• Skill Lead Hadoop Spark Developer
• Primary Skill Python, Pyspark, Spark, SQL, Unix Shell Scripting, Airflow (Highlighted in Red are mandate)
• Exp 10- 15 Yrs
• Location Chennai (Sholinganallur SEZ) / Bangalore (RMZ Ecoworld Bellandur)
• Hybrid Mode 3 Days a week as per organization policy

Job Title: Data Engineering Hadoop Developer (10-15 Years Experience)

Job Summary

Seeking a Hadoop Developer with 8 years of experience in big data engineering to design, build, and optimize scalable data pipelines on Hadoop ecosystem technologies. The role involves batch/stream processing, data integration, performance tuning, and close collaboration with business teams.

Key Responsibilities

1. Design and develop robust ETL/data pipelines using Hadoop ecosystem tools.
2. Build and optimize large-scale data processing jobs using Spark and/or Hive.
3. Develop workflows using Airflow.
4. Write complex SQL/HiveQL for data transformation and reporting needs.
5. Ingest structured and unstructured data from multiple sources (Kafka, Sqoop, APIs, files).
6. Ensure data quality, lineage, and governance standards are followed.
7. Monitor, troubleshoot, and tune jobs for performance and scalability.
8. Create technical documentation and follow coding best practices.

Required Skills

1. 8 years of hands‑on experience in Hadoop/big data development.
2. Strong knowledge of HDFS, Hive, Spark,
3. String knowledge in Python/Scala
4. Strong SQL and data modelling fundamentals.
5. Experience with workflow orchestration tools (Airflow).
6. Good understanding of Linux, shell scripting, and distributed systems.
7. Familiarity with version control (Git) and Agile delivery practices.

Tech Mahindra represents the connected world, offering innovative and customer‑centric information technology experiences, enabling Enterprises, Associates and the Society to Rise . We are a USD 4.9 billion company with 121,840+ professionals across 90 countries, helping over 935 global customers including Fortune 500 companies. Our convergent, digital, design experiences, innovation platforms and reusable assets connect across a number of technologies to deliver tangible business value and experiences to our stakeholders. Tech Mahindra is the highest ranked Non-U.S. company in the Forbes Global Digital 100 list (2018) and in the Forbes Fab 50 companies in Asia (2018).