AWS Data Engineer

Tata Consultancy Services; Bengaluru Urban; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tata Consultancy Services in Bengaluru is seeking a hands-on data engineer to design and optimize large-scale data pipelines. You will work with Spark, Hive, Oozie on AWS and on-prem Cloudera, ensuring reliable data flows and scalable processing.

The role requires strong scripting in Scala/Java/Python, experience with Hadoop ecosystem, and a bias for testing, performance tuning, and source control with GitLab.

Qualifications

• Proficiency in Scala/Java/Python.
• Hands-on experience with Oozie, Hive and Scala.
• Experience with Apache Spark.
• Cloud expertise: AWS.
• Hands-on building on-prem data pipelines using Cloudera, including Hive and Oozie.
• Performance tuning techniques for Hive and Scala.
• Unit testing.
• Familiarity with GitLab for source control.

Skills

Scala

Java

Python

Oozie

Hive

Spark

AWS

Cloudera

Hive tuning

GitLab

Unit tests

Tools

GitLab

Cloudera

Job description

Key Requirements/ Past Experiences/ Knowledge

• Proficiency in Scala/Java/Python
• Oozie, Hive & Scala (hands on)
• Experience with Apache Spark
• Cloud expertise: primarily AWS
• Hands-on experience in building on-premises data pipelines using big data platforms like Cloudera, including basic Hive and Oozie
• Performance tuning techniques - for hive & scala
• Unit testing
• Familiarity with GitLab for source control

Location