Big Data & Hadoop Developer

Tata Consultancy Services Limited; Pune District, Chennai, Bengaluru; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Tata Consultancy Services Limited is seeking a Big Data & Hadoop Developer for multiple locations including Chennai, Pune, Kolkata, Bangalore, and Hyderabad. The role requires 6-10 years of experience in Java/Scala, Spark, Hadoop, and related SQL tools.

You will build scalable data transformation applications and contribute to CI/CD pipelines while working with a distributed team. Responsibilities include developing Spark-based solutions, working with HDFS and Hive SQL, and ensuring code quality

Qualifications

• Hands-on experience with JAVA/SCALA in Spark-based data pipelines.
• Proficiency with Unix shell scripting and HDFS commands.
• Strong SQL knowledge across Hive, Spark SQL, and Impala.

Responsibilities

• Develop and maintain scalable data transformation applications using Apache Spark.
• Contribute to code in Scala/Java and integrate with CI/CD pipelines.
• Provide Spark-based solutions for large data with Hive/MPP like Impala.
• Create Junit tests and ensure code coverage per standards.
• Review code modules from juniors and collaborate with distributed teams.

Skills

JAVA

SCALA

Spark Frameworks

Unix Shell scripting

Tools

Maven

Apache Spark

Hadoop

Hive

HBase

Spark SQL

Impala

Job description

Role: Big Data & Hadoop Developer

Location:Chennai , Pune, Kolkata , Bangalore, Hyderabad

Experience: 6-10yrs

Responsibility:

• Hands on development experience in programming languages such as JAVA,SCALA using Maven, Apache Spark Frameworks and Unix Shell scripting
• Should be comfortable with Unix File system as well as HDFS commands Should have worked on query languages such as Oracle SQL, Hive SQL, Spark SQL, Impala, HBase DB
• Should be flexible Should have good communication and customer management skills
• Design high quality deliverables adhering to business requirement with defined standards and design principles, patterns
• Develop and maintain highly scalable, high performance Data transformation applications using Apache Spark framework
• Develop/Integrate the code adhering to CI/CD, using Spark Framework in Scala/Java
• Provide solutions to Big data problems dealing with huge volumes of data using Spark based data transformation solutions , Hive, MPP processes like IMPALA.
• Create Junit tests and ensure code coverage is met as per the agreed standards
• Should be able to work with a team who might be geographically distributed. Review the code modules developed by other juniors.