Data Engineer

Tata Consultancy Services Limited; Pune District; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Tata Consultancy Services Limited in Pune is seeking a hands-on Data Engineering professional with 5–10 years of experience. The role emphasizes DataProc, Composer, Scala, and PySpark, with strong Hadoop ecosystem exposure including Spark, Hive, and NoSQL databases.

The candidate will work on end-to-end data engineering tasks, collaborate with cross-functional teams, and contribute to high-quality, timely deliverables in a dynamic cloud-based environment.

Qualifications

• 5–10 years of experience in data engineering or similar roles.
• Proficient in DataProc, Composer, Scala and PySpark.
• Experience with Hadoop ecosystem and Spark, Hive, NoSQL databases.

Responsibilities

• Excellent communication (oral and written) and documentation skills.
• Hands-on experience with Hadoop distributions (MapR, Hortonworks, Cloudera) and related tools.
• Hands-on experience with Google Cloud Platform services: Cloud Storage, DataFlow, DataProc, BigQuery, Pub/Sub, etc.
• Broad involvement in analysis, design, development, and testing as needed.
• Self-starter, team player, and commitment to quality and time-to-market.

Skills

DataProc

Composer

Scala

PySpark

Tools

Spark

Hadoop

Hive

HBase

Cassandra

MongoDB

Job description

Experience: 5-10 Years

Must-Have:

• DataProc
• Composer
• Scala
• PySpark

Responsibility:

• Excellent communication (both oral and written) and documentation
• Hands-on experience in the complete eco-system of any one/more of the various Hadoop distributions such as MapR (preferred) and/or Hortonworks and/or Cloudera and/or Apache Hadoop, with specialization in tools such as Spark (Core and SQL), Hive and NoSql Databases such as HBase and/or Cassandra and/or MongoDB
• Hands-on experience in the various services of Google Cloud Platform (GCP) cloud such as Cloud Storage, DataFlow, DataProc, CI/CD, Big Query, Spanner, BigTable, Composer, Pub/Sub, Hyper-drive, etc…
• Broader responsibilities include but not limited to analysis, design, development and testing as needed
• Should be a self-starter, team player, collaborative and should be thorough focusing on the quality of the deliverable and time to market