Big Data Engineer

Tata Consultancy Services; Pune District; On-site; INR 1,400,000 - 2,100,000; Full time

Job summary

Tata Consultancy Services in Pune is seeking an experienced Big Data Engineer to design, develop, and optimize large-scale data processing solutions. You will build high-performance data pipelines, manage distributed systems, and enable scalable analytics platforms.

Required skills include Spark, Scala, Hadoop ecosystem components, SQL/data modeling, and ETL pipeline development, with preferred experience in NiFi, Kafka, AWS, Airflow, and CI/CD practices.

Qualifications

• Experience designing and implementing large-scale data processing solutions.
• Strong background in building robust data pipelines and data models.
• Knowledge of distributed systems, performance tuning, and reliability best practices.

Responsibilities

• Design, develop and optimize large-scale data processing solutions.
• Build high-performance data pipelines and data integration workflows.
• Collaborate with data science and analytics teams to enable scalable analytics.

Skills

Apache Spark

Scala

SQL & Data Modeling

Distributed Computing Concepts

ETL/Data Pipeline Development

Tools

Hadoop Ecosystem (HDFS, Hive, Yarn)

Apache NiFi

Apache Kafka

AWS Services

Airflow

CI/CD & DevOps

Job description

• Seeking an experienced Big Data Engineer with strong expertise in Spark, Scala, and Hadoop ecosystem technologies to design, develop, and optimize large-scale data processing solutions.
• The candidate will be responsible for building high-performance data pipelines, managing distributed systems, and enabling scalable analytics platforms.
• Required Skills: Apache Spark, Scala, Hadoop Ecosystem (HDFS, Hive, Yarn), SQL & Data Modeling, Distributed Computing Concepts, ETL/Data Pipeline Development
• Preferred Skills: Apache NiFi, Apache Kafka, AWS Services, Airflow, CI/CD & DevOps