Walk-in | Spark with (Scala or Python)

Tata Consultancy Services; Chennai District; On-site; INR 1,000,000 - 1,600,000; Full time

Job summary

Tata Consultancy Services in Chennai seeks a Spark developer with hands-on Java/Scala experience, Maven, and strong Spark framework skills. The role requires Unix shell scripting, comfort with Unix/HDFS and expertise in Spark SQL or Hive SQL, Impala, and HBase.

You will design, build, test, and maintain scalable data transformation solutions and collaborate across distributed teams. The ideal candidate communicates well and can manage customer interactions, contributing to high-quality data

Qualifications

• Hands on development experience in Java/Scala using Maven, Apache Spark frameworks and Unix shell scripting.
• Comfort with Unix file system and HDFS commands.
• Experience with SQL variants including Oracle, Hive, Spark SQL, Impala and HBase.
• Good communication and customer management skills.

Responsibilities

• Design high quality deliverables adhering to business requirements and standards.
• Develop and maintain scalable, high performance data transformation apps using Apache Spark.
• Develop/integrate code with CI/CD using Spark framework in Scala/Java.
• Provide Spark-based data transformation solutions with Hive and Impala for large data volumes.
• Create JUnit tests and ensure code coverage per standards.
• Collaborate with geographically distributed teams and review juniors' modules.

Skills

JAVA/SCALA development

Maven

Unix Shell scripting

Spark Frameworks

Tools

Apache Spark SQL

Hive SQL

Spark SQL

Impala

HBase

Oracle SQL

HDFS commands

Job description

Mode - F2F

Skill - Spark with (Scala or Python)

Location - Chennai

Must-Have:

• Hands on development experience in programming languages such as JAVA,SCALA using Maven, Apache Spark Frameworks and Unix Shell scripting
• Should be comfortable with Unix File system as well as HDFS commands
• Should have worked on query languages such as Oracle SQL, Hive SQL, Spark SQL, Impala, HBase DB
• Should be flexible Should have good communication and customer management skills

Role & responsibilities

• Design high quality deliverables adhering to business requirement with defined standards and design principles, patterns
• Develop and maintain highly scalable, high performance Data transformation applications using Apache Spark framework
• Develop/Integrate the code adhering to CI/CD, using Spark Framework in Scala/Java
• Provide solutions to Big data problems dealing with huge volumes of data using Spark based data transformation solutions , Hive, MPP processes like IMPALA.
• Create Junit tests and ensure code coverage is met as per the agreed standards
• Should be able to work with a team who might be geographically distributed. Review the code modules developed by other juniors.