Big Data Developer

Tata Consultancy Services Limited; Bengaluru; On-site; INR 1,800,000 - 2,400,000; Full time

Job summary

Tata Consultancy Services Limited is seeking a Big Data Developer/Engineer in Bengaluru to design and develop scalable data solutions. The role requires strong hands-on experience with Scala, Apache Hadoop, PySpark, and Hive, and involves building and optimizing large-scale data pipelines using Spark, Spark SQL/HiveQL, and Hadoop ecosystem components.

You'll work on ETL/ELT pipelines, ensure data quality and scalability, and may leverage Airflow, Git, and CI/CD in a cloud-enabled environment

Qualifications

• 4-15 years of experience in designing and developing scalable data solutions.
• Strong hands-on experience in Scala, Hadoop, PySpark, and Hive.
• Designing and developing large-scale data processing pipelines using Spark/PySpark and Scala.
• Deep experience with the Hadoop ecosystem (HDFS, Hive, YARN).
• Proficient in HiveQL/SQL for data extraction, transformation and analysis.
• Building robust ETL/ELT pipelines for high-volume datasets.
• Spark job performance tuning, troubleshooting and optimization.
• Experience with Linux/Unix, Git, CI/CD, and orchestration tools (Airflow) is desirable.
• Exposure to AWS, Azure or GCP and modern data platforms.

Responsibilities

• Design and develop scalable data solutions.
• Develop and optimize large-scale data pipelines using Spark, PySpark and Scala.
• Work with Hadoop ecosystem components (HDFS, Hive, YARN).
• Write HiveQL/SQL for data extraction and transformation.
• Build and maintain ETL/ELT pipelines for high-volume data.
• Tune and troubleshoot Spark jobs for performance and reliability.
• Ensure data quality, reliability, and availability across workflows.
• Collaborate with data architects, analysts, and engineering teams.

Skills

Scala

Hadoop

PySpark

Hive

HiveQL/SQL

ETL/ELT

Spark tuning

Linux/Unix

Tools

Airflow

Git

CI/CD

Job description

Date Of Interview: 31-Oct,2026

Mode of Interview: Virtual

Responsibilities :-

• Seeking a Big Data Developer/Engineer with 4-15 years of experience in designing and developing scalable data solutions.
• Strong hands-on experience in Scala, Apache Hadoop, PySpark, and Hive is mandatory.
• Develop and optimize large-scale data processing pipelines using Spark/PySpark and Scala.
• Work extensively with the Hadoop ecosystem, including HDFS, Hive, YARN, and related Big Data technologies.
• Write efficient HiveQL/SQL queries for data extraction, transformation, aggregation, and analysis.
• Build robust ETL/ELT pipelines to process high-volume structured and semi-structured datasets.
• Perform Spark job performance tuning, troubleshooting, and optimization for large datasets.
• Ensure data quality, reliability, scalability, and availability across Big Data processing workflows.
• Experience with Linux/Unix, Git, CI/CD, and scheduling/orchestration tools such as Apache Airflow is desirable.
• Exposure to cloud platforms such as AWS, Azure, or GCP and modern data platforms is an added advantage.
• Collaborate with data architects, analysts, and engineering teams to deliver production-ready Big Data solutions.