Big Data Engineer (Spark & Scala)

Tata Consultancy Services Limited; Pune District, Chennai; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Tata Consultancy Services Limited is seeking a Big Data Engineer with strong Scala and Apache Spark expertise to design, develop, and optimize large-scale data processing solutions. The role involves building robust data pipelines and working with distributed platforms across Chennai and Pune.

Ideal candidates will have hands-on experience with Hadoop, SQL, cloud platforms (Azure/AWS/GCP), and version control.

Qualifications

• 6-8 years of experience in Data Engineering and Big Data technologies.
• Strong proficiency in Scala and Apache Spark.
• Hands-on experience with Hadoop ecosystem components.
• Strong SQL and data modeling skills.
• Experience with cloud platforms (Azure/AWS/GCP).
• Familiarity with distributed computing concepts and performance tuning.
• Experience with version control systems such as Git.

Responsibilities

• Design and develop scalable big data applications using Scala and Apache Spark.
• Build and maintain batch and real-time data processing pipelines.
• Optimize Spark applications for performance and scalability.
• Develop ETL/ELT workflows and data integration solutions.
• Collaborate with cross-functional teams to translate business requirements into technical solutions.
• Troubleshoot and resolve performance and data quality issues.
• Ensure adherence to coding standards and best practices.

Skills

Scala

Apache Spark

Hadoop ecosystem

SQL

Cloud platforms

Git

Distributed computing

Education

Bachelor's or Master's in CS/IT or related field

Tools

Databricks

Kafka

Delta Lake

Spark Streaming

Airflow

Docker

Kubernetes

CI/CD

Job description

Big Data Engineer (Spark & Scala)

Job Title: Big Data Engineer - Spark & Scala

Experience: 6-8 Years
Location: Chennai / Pune
Employment Type: Full-Time

Job Description

We are looking for an experienced Big Data Engineer with strong expertise in Scala and Apache Spark to design, develop, and optimize large-scale data processing solutions. The ideal candidate should have hands‑on experience building robust data pipelines and working with distributed data platforms.

Key Responsibilities

• Design and develop scalable big data applications using Scala and Apache Spark.
• Build and maintain batch and real-time data processing pipelines.
• Optimize Spark applications for performance and scalability.
• Develop ETL/ELT workflows and data integration solutions.
• Collaborate with cross-functional teams to translate business requirements into technical solutions.
• Troubleshoot and resolve performance and data quality issues.
• Ensure adherence to coding standards and best practices.

Required Skills

• 6-8 years of experience in Data Engineering and Big Data technologies.
• Strong proficiency in Scala and Apache Spark.
• Hands‑on experience with Hadoop ecosystem components.
• Strong SQL and data modeling skills.
• Experience with cloud platforms (Azure/AWS/GCP).
• Familiarity with distributed computing concepts and performance tuning.
• Experience with version control systems such as Git.

Preferred Skills

• Experience with Databricks, Kafka, Delta Lake, or Spark Streaming.
• Exposure to Airflow, Docker, and Kubernetes.
• Knowledge of CI/CD pipelines and DevOps practices.

Education

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

Keywords

Spark, Scala, Big Data, Hadoop, Kafka, Databricks, Data Engineering, ETL, Azure, AWS, GCP, Spark Streaming, Delta Lake

#BigData #Spark #Scala #DataEngineer #Hiring #ChennaiJobs #PuneJobs #NaukriJobs #TechHiring #DataEngineering