Data Engineer (Spark and Scala)

Tata Consultancy Services; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tata Consultancy Services in Bengaluru is hiring a Data Engineer skilled in Spark and Scala to lead data pipelines and production deployment.

Must-have skills include Hadoop ecosystem, Scala, Spark, HDFS, Hive, Impala, Sqoop, Java; experience in data ingestion and building data products. You will lead a team of 3–5 engineers, coordinating with analytics teams to deliver data solutions.

Qualifications

• 6–10 years of IT experience in data engineering or related roles.
• Strong knowledge of Spark, Scala, and Hadoop ecosystem.
• Experience with data ingestion pipelines using Sqoop, HDFS, Hive and Impala.
• Proficient in SQL and Unix commands.

Responsibilities

• Lead Data Engineer to build data pipelines supporting data science and analytics use cases.
• Develop code in the required language, test it, and deploy to Production.
• Contribute to solution design and requirements analysis.
• Provide technical leadership to a team of 3–5 data engineers.
• Coordinate with wider IT and business teams to integrate data products.

Skills

Scala

Spark

Hadoop

HDFS

Hive

Impala

Sqoop

Java

Education

15 years of full-time education

Tools

Pyspark

Tableau

SAS

Job description

TCS Hiring for Data Engineer (Spark and Scala)

Experience Range: - 06 To 10 Years (Mandatory)

(Note: Candidates below 6 years of IT experience shall not be considered)

Job Locations : Bengaluru, Chennai, Hyderabad, Pune, Delhi

JOB DESCRIPTION

Must Have Skills:

• Big Data/Hadoop Experience particularly in ingesting data and implementing Data ingestion pipelines, SQOOP, HADOOP, HDFS, HIVE, IMPALA, Java, Scala, Spark
• Scala is Mandatory

Data Engineer with below responsibilities:

• Lead Data Engineer to build data pipelines to support implementation of data science and analytics use cases.
• Candidate needs to be able to develop code in the corresponding language (see technical skills), test it, and follow up with the implementation into Production environment.
• Candidate will also take part in the solution design phase, so experience in analysis requirements is desirable.
• Technical expert to lead a squad of engineers for a Data Product and implement data transformation projects in Hadoop.
• Candidate will lead a team of 3-5 data engineers with minimum supervision and integrate with wider IT and business project teams.

Good to Have Skills:

• Good exposure to Unix and HDFS commands
• Experience on Pyspark will be an added advantage
• Experience working on Data Analytics will be an added advantage
• Experience on SAS will be an added advantage

Responsibilities or Expectations from the Role:

• Good work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQL
• Experience working on Data Engineering projects
• Good Understanding of SQL
• Good understanding of Unix and HDFS commands
• Communication
• Experience working on Data Analytics and Pyspark
• Exposure to Tableau & SAS

15 years of Full Time Education