Bigdata developer

Tata Consultancy Services; Chennai District; On-site; INR 1,200,000 - 2,000,000; Full time

Job summary

Tata Consultancy Services is hiring for a Big Data Engineer in Chennai to work on Hadoop ecosystem, Spark, PySpark, Hive and related tools. You will develop and optimize large-scale data processing pipelines, write Spark jobs using Python, and collaborate with cross-functional teams.

Strong debugging, software design skills, and experience in SDLC are required. Familiarity with Autosys or Unix scripting is a plus, with intent to deliver robust data solutions for client engagements.

Qualifications

• Good understanding of Hadoop concepts including file system and Map Reduce.
• Hands on experience on Spark framework, Unix scripting, Hive queries, wriring UDF in Hive. Theortical knowledge and POC alone will not suffice.
• Strong debugging skills in Hive and Spark; experience developing large scale systems; client-facing collaboration.

Skills

PySpark

HDFS

Hive

Spark

Sqoop

Flume

Oozie

Unix Script

Autosys

Tools

Python

Job description

Required Technical Skill Set**Big Data – PySpark, HDFS, Hive, Spark, Sqoop, Flume, Oozie, Unix Script, Autosys

Desired Competencies (Technical/Behavioral Competency)

Must-Have**

• Good understanding of Hadoop concepts including file system and Map Reduce.


• Hands on experience on Spark framework, Unix scripting, Hive queries, wriring UDF in Hive. Theortical knowledge and POC alone will not suffice.


• Good knowledge in Software Development Life Cycle and Project Development Lifecycle.


• Associate should be able to work independently and should have strong debugging skill in both Hive and Spark. Associate should have experience developing large scale systems, experience debugging and performance tuning, excellent software design skills, communication skills and ability to work with client partners. Hands on in Apache Spark with Python or any other language (Preferred is Python -PySpark)

Good-to-Have

• Experience in Banking and Finance Domain.


• Experience in Agile Methodology


• Knowledge in job scheduling tools like Autosys.