Hiring Bigdata Engineer - Chennai/ Bangalore

Tech Mahindra Limited; Chennai District; Hybrid; INR 900,000 - 1,500,000; Full time

Job summary

Tech Mahindra is seeking a Data Engineering Hadoop Developer to design, build, and optimize scalable data pipelines on Hadoop ecosystem technologies in India. The role covers batch and streaming processing, data integration, and performance tuning with collaboration across business teams.

The candidate should have 3–8 years of hands-on Hadoop/big data development experience, strong SQL and data modelling skills, and proficiency in Python or Scala.

Qualifications

• 3–8 years of hands-on experience in Hadoop/big data development.
• Strong knowledge of HDFS, Hive and Spark.
• Proficient in Python and/or Scala.
• Solid SQL skills and data modelling fundamentals.
• Experience with Airflow and data pipelines.
• Familiarity with Linux, shell scripting, and distributed systems.
• Version control with Git and Agile delivery practices.

Responsibilities

• Design and develop robust ETL/data pipelines using Hadoop ecosystem tools.
• Build and optimize data processing jobs with Spark and Hive.
• Develop workflows using Airflow.
• Write SQL/HiveQL for transformations and reporting needs.
• Ingest structured and unstructured data from multiple sources (Kafka, Sqoop, APIs, files).
• Ensure data quality, lineage, and governance standards are followed.
• Monitor, troubleshoot, and tune jobs for performance and scalability.
• Create technical documentation and follow coding best practices.

Skills

Hadoop/big data development

HDFS

Hive

Spark

Python

Scala

SQL

Airflow

Linux

Shell scripting

Git

Agile delivery

Job description

Data Engineering Hadoop Developer (3-8 Years Experience)

Seeking a Hadoop Developer with 3-8 years of experience in big data engineering to design, build, and optimize scalable data pipelines on Hadoop ecosystem technologies. The role involves batch/stream processing, data integration, performance tuning, and close collaboration with business teams.

Key Responsibilities

• Design and develop robust ETL/data pipelines using Hadoop ecosystem tools.
• Build and optimize large‑scale data processing jobs using Spark and/or Hive.
• Develop workflows using Airflow.
• Write complex SQL/HiveQL for data transformation and reporting needs.
• Ingest structured and unstructured data from multiple sources (Kafka, Sqoop, APIs, files).
• Ensure data quality, lineage, and governance standards are followed.
• Monitor, troubleshoot, and tune jobs for performance and scalability.
• Create technical documentation and follow coding best practices.

Required Skills

• 3‑8 years of hands‑on experience in Hadoop/big data development.
• Strong knowledge of HDFS, Hive, Spark.
• Strong knowledge in Python/Scala.
• Strong SQL and data modelling fundamentals.
• Experience with workflow orchestration tools (Airflow).
• Good understanding of Linux, shell scripting, and distributed systems.
• Familiarity with version control (Git) and Agile delivery practices.