Spark/Scala Data Engineer

Tata Consultancy Services; Chennai District; On-site; INR 800,000 - 1,400,000; Full time

Job summary

Tata Consultancy Services in India seeks professionals with hands-on experience in Java/Scala, Maven and Spark, plus strong SQL skills across Oracle, Hive, Spark SQL, Impala and HBase. Proficiency in Unix shell scripting and Big Data ecosystem familiarity is required.

Proven ability to work on data ingestion and data processing pipelines, with experience using Eclipse IDE and Oracle SQL Developer. Strong communication and client management skills are essential.

Qualifications

• Hands-on development in Java/Scala with Maven and Spark.
• Unix shell scripting and Unix file system/HDFS familiarity.
• Experience with Oracle SQL, Hive SQL, Spark SQL, Impala and HBase.
• Familiar with Eclipse IDE and Oracle SQL Developer.
• Strong communication and customer management skills.
• Knowledge of Big Data ecosystem components.

Responsibilities

• Work on big data projects end-to-end.
• Collaborate with cross-functional teams to deliver data solutions.
• Ingest data using tools like Sqoop and Kafka.
• Analyze and optimize SQL queries across Hive, Spark and Oracle.

Skills

Java

Scala

Maven

Apache Spark

Unix Shell scripting

Oracle SQL

Hive SQL

Spark SQL

Impala

HBase

SQL

Tools

Eclipse IDE

Tectia Client

Oracle SQL Developer

Job description

Hands on development experience in programming languages such as JAVA,SCALA using Maven, Apache Spark Frameworks and Unix Shell scripting

Should be comfortable with Unix File system as well as HDFS commands

Should have worked on query languages such as Oracle SQL, Hive SQL, Spark SQL, Impala, HBase DB

Should be flexible

Should have good communication and customer management skills

Should have knowledge on Big data Data Ingestion tools such as SQOOP and KAFKA.

Should be aware of the components in Big Data ecosystem.

Should have worked on building projects using Eclipse IDE, Tectia Client, Oracle SQL Developer.