Big Data Developer

Tata Consultancy Services; Bengaluru; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Tata Consultancy Services in Bengaluru is seeking a skilled data engineer to design and develop scalable data transformation applications using Apache Spark with Scala/Java. You will ensure code quality, write tests, and collaborate with a distributed team.

Strong SQL skills across Oracle/Hive/Spark/Impala are essential to transform large datasets efficiently. The role requires 5–7 years of relevant experience, a technical degree, and good communication to liaise with stakeholders.

Qualifications

• Hands on development experience in JAVA/SCALA using Maven and Spark frameworks.
• Experience with Unix shell scripting, Unix file system and HDFS commands.
• Proficient in Oracle SQL, Hive SQL, Spark SQL, Impala and HBase.
• Good communication and customer management skills.

Responsibilities

• Design high-quality deliverables adhering to standards and design principles.
• Develop scalable, high-performance data transformation applications with Spark.
• Integrate code with CI/CD using Spark in Scala/Java.
• Provide solutions for big data problems using Spark, Hive, and IMPALA.
• Create JUnit tests and ensure code coverage; review modules from juniors.

Skills

Java

Scala

Maven

Apache Spark

Unix Shell scripting

Unix File system

HDFS commands

Oracle SQL

Hive SQL

Spark SQL

Impala

HBase

Communication skills

Education

BE/BTech/MTech/MCA/BCA/BSc/Diploma/Masters

Tools

Eclipse IDE

Tectia Client

Oracle SQL Developer

Job description

Responsibilities:

• Design high-quality deliverables adhering to business requirements with defined standards and design principles, patterns
• Develop and maintain highly scalable, high-performance Data transformation applications using Apache Spark framework
• Develop/Integrate the code adhering to CI/CD, using Spark Framework in Scala/Java
• Provide solutions to big data problems dealing with huge volumes of data using Spark based data transformation solutions, Hive, MPP processes like IMPALA.
• Create Junit tests and ensure code coverage is met as per the agreed standards
• Should be able to work with a team who might be geographically distributed. Review the code modules developed by other juniors.

Must Have Skills:

• Hands on development experience in programming languages such as JAVA, SCALA using Maven, Apache Spark Frameworks and Unix Shell scripting.
• Should be comfortable with Unix File system as well as HDFS commands.
• Should have worked on query languages such as Oracle SQL, Hive SQL, Spark SQL, Impala, HBase DB.
• Good communication and customer management skills

Good to have Skills

• Knowledge on Big data Data Ingestion tools such as SQOOP and KAFKA.
• Should be aware of the components in Big Data ecosystem.
• Should have worked on building projects using Eclipse IDE, Tectia Client, Oracle SQL Developer.

Experience Range: 5-7 yrs

Qualifications:

Any Technical Degree (BE/B,Tech/M.tech/MCA/BCA/BSC/Diploma/Masters)

• 15 years of full-time education.
• Good communication and interpersonal skills
• Strong verbal and written communication skills
• Willingness to work in a collaborative and dynamic environment
• Relevant experience as per the job requirements (if applicable)

https://www.tcs.com/careers/india/tcs-data-and-analytics-careers