Walk-in | Spark Scala Developer

Tata Consultancy Services; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tata Consultancy Services is seeking a hands-on Big Data engineer with 4-10 years of experience for Pan India locations. The role focuses on Java/Scala development with Maven, Apache Spark, and Spark SQL, leveraging HDFS and Hive/Impala.

You will design and implement scalable data transformation pipelines, ensure CI/CD, and collaborate with a geographically distributed team to deliver high-quality data solutions.

Qualifications

• Hands on development experience in Java/Scala using Maven and Spark frameworks.
• Comfortable with Unix File system as well as HDFS commands.
• Proficient in query languages: Oracle SQL, Hive SQL, Spark SQL, Impala, HBase.
• Design high quality data solutions adhering to standards and design principles.
• Develop and maintain scalable data transformation apps using Spark.
• Implement CI/CD with Spark in Scala/Java.
• Provide solutions to big data problems using Spark, Hive, and Impala.
• Create JUnit tests and ensure code coverage.
• Collaborate with distributed teams and review junior developers' work.

Responsibilities

• Develop and maintain scalable data transformation apps using Spark.
• Adhere to CI/CD pipelines for code integration.
• Create and maintain unit tests to ensure code quality and coverage.
• Review modules and mentor junior developers; collaborate with distributed teams.

Skills

JAVA

SCALA

Unix Shell scripting

Apache Spark

CI/CD

Maven

Spark Frameworks

Spark SQL

Oracle SQL

Hive SQL

Impala

HBase

Junit tests

Tools

HDFS

Junit

Job description

Experience:4-10Years

Location Pan India(Metro cities):

• Hands on development experience in programming languages such as JAVA,SCALA using Maven, Apache Spark Frameworks and Unix Shell scripting
• Should be comfortable with Unix File system as well as HDFS commands
• Should have worked on query languages such as Oracle SQL, Hive SQL, Spark SQL, Impala, HBase DB
• Design high quality deliverables adhering to business requirement with defined standards and design principles, patterns
• Develop and maintain highly scalable, high performance Data transformation applications using Apache Spark framework
• Develop/Integrate the code adhering to CI/CD, using Spark Framework in Scala/Java
• Provide solutions to Big data problems dealing with huge volumes of data using Spark based data transformation solutions , Hive, MPP processes like IMPALA.
• Create Junit tests and ensure code coverage is met as per the agreed standards
• Should be able to work with a team who might be geographically distributed. Review the code modules developed by other juniors.