Walk-in | Big Data Engineer - Spark & Scala

Tata Consultancy Services; Chennai District; On-site; INR 700,000 - 1,100,000; Full time

Job summary

Tata Consultancy Services is seeking a Big Data Engineer in Chennai to design and implement scalable data transformation solutions using Spark, Hive, and Unix shell scripting. The role emphasizes building high-quality deliverables aligned with business requirements and standards.

You will work with Scala/Java-based Spark applications, create JUnit tests, ensure code coverage, and collaborate with geographically distributed teams.

Qualifications

• Design high quality deliverables as per business requirements and design principles.
• Develop and maintain scalable, high-performance data transformation applications using Apache Spark.
• Develop/integrate code adhering to CI/CD using Spark framework in Scala/Java.
• Provide solutions to big data problems with Spark-based transformations, Hive, and IMPALA-compatible MPP processes.
• Create JUnit tests and ensure code coverage per standards.
• Collaborate with distributed teams and review modules developed by juniors.

Responsibilities

• Design high quality deliverables adhering to business requirement with defined standards and design principles, patterns
• Develop and maintain highly scalable, high performance Data transformation applications using Apache Spark framework
• Develop/Integrate the code adhering to CI/CD, using Spark Framework in Scala/Java
• Provide solutions to Big data problems dealing with huge volumes of data using Spark based data transformation solutions , Hive, MPP processes like IMPALA.
• Create Junit tests and ensure code coverage is met as per the agreed standards
• Should be able to work with a team who might be geographically distributed. Review the code modules developed by other juniors.

Skills

Big data

Scala/Java

Python

Spark

Hive

Unix Shell Scripting

Job description

Required Technical Skill Set**

Bigdata Scala/Java, Python, Spark, Hive, Unix Shell Scripting

Responsibility of / Expectations from the Role

1 Design high quality deliverables adhering to business requirement with defined standards and design principles, patterns

2 Develop and maintain highly scalable, high performance Data transformation applications using Apache Spark framework

3 Develop/Integrate the code adhering to CI/CD, using Spark Framework in Scala/Java

4 Provide solutions to Big data problems dealing with huge volumes of data using Spark based data transformation solutions , Hive, MPP processes like IMPALA.

5 Create Junit tests and ensure code coverage is met as per the agreed standards

6 Should be able to work with a team who might be geographically distributed. Review the code modules developed by other juniors.