Big Data Developer

Tata Consultancy Services; Pune District; On-site; INR 1,200,000 - 1,600,000; Full time

Job summary

Tata Consultancy Services in Pune, India, is seeking a data engineer to design and implement scalable data pipelines using HDFS, Hive/HQL, Spark, and Kafka. The role requires contributing to the development of high-performance data processing solutions within an Agile team.

The associate will participate in scrum ceremonies, troubleshoot issues, and ensure delivery of quality data products aligned with business needs.

Qualifications

• Must-have technical competencies in big data technologies as listed in the ad.
• Proficient with HDFS, Hive/HQL and Spark for data processing.
• Experience with real-time streaming or messaging systems like Kafka is expected.
• Ability to work with Arcadia and Shell scripting to support pipelines.

Responsibilities

• Design high quality deliverables adhering to business requirements and standards.
• Develop and maintain highly scalable, high performance data pipelines.
• Attend scrum meetings and collaborate with teams to ensure timely delivery.

Skills

HDFS

Hive/HQL

Shell Scripting

Spark

Arcadia

Kafka

Cassendra

Job description

Desired Competencies (Technical/Behavioral Competency)

Must-Have

HDFS, Hive/HQL, Shell Scripting, Spark, Arcadia, Kafka, Cassendra

Good-to-Have

Should be good in troubleshooting and Agile experience is must

Responsibility of / Expectations from the Role

• 1.Design high quality deliverables adhering to business requirement with defined standards and design principles, patterns
• 2.Develop and maintain highly scalable, high performance and highly available
• 3.Associate should be available and should attend the scrum meetings without Fail