Big Data Engineer

Tata Consultancy Services; Bengaluru; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Tata Consultancy Services in Bengaluru is seeking a Big Data Developer with 6 to 8 years of hands-on experience in Hadoop ecosystem, including HDFS, Hive, Spark, Sqoop, Flume, Oozie, Unix scripting and Autosys. The role requires strong data transformation, SQL skills and collaboration.

You will gather requirements, design and develop data ingestion frameworks using Spark, Hive and Sqoop, and contribute to ETL and BI concepts. Excellent communication and teamwork are essential.

Qualifications

• Hands-on experience in Big data, Hadoop.
• Hands-on experience in Hive, Sqoop, Spark, Hive.
• Hands on experience with Data Transformations, data structures, Metadata, SQL, workload management.
• Coding experience in SQL.
• Good communication skills and team skills.
• Effective communication and independently.

Responsibilities

• Requirements gathering and data analysis for multiple business processes.
• Development of Data ingestion framework using Spark, Hive, Sqoop tech stack.
• Good analytical ability.
• Good understanding of ETL and BI concepts.

Skills

Big Data

Hadoop

Data transformations

Metadata management

SQL proficiency

Communication

Teamwork

Independent work

Tools

HDFS

Hive

Spark

Sqoop

Flume

Oozie

Unix scripting

Autosys

SQL

Job description

Desired experience: 6 to 8 years

Required Technical Skill Set: HDFS, Hive, Spark, Sqoop, Flume, Oozie, Unix Script, Autosys

Must-Have:

• Hands-on experience in Big data, Hadoop
• Hands-on experience in Hive, Sqoop, Spark, Hive
• Hands on experience with Data Transformations, data structures, Metadata, SQL, workload management
• Coding experience in SQL
• Good communication skills and team skills
• Effective communication and independently

Responsibilities

• As a Developer, must be able to do Requirements gathering and data analysis for the multiple business processes involved
• Development of Data ingestion framework using Spark, Hive, Sqoop tech stack
• Good analytical ability
• Good understanding of ETL and BI concepts