Hadoop Developer

LTM; Hyderabad; Presencial; INR 900.000 - 1.800.000; Jornada completa

Descripción de la vacante

LTM in Hyderabad, India, is seeking a skilled Big Data Engineer to develop and maintain large-scale data processing applications using Scala and Apache Spark. You will design and optimize Spark SQL jobs for batch and real-time data processing.

You will work with Hadoop components such as HDFS, Hive, YARN, and MapReduce, build ETL pipelines, and perform performance tuning to ensure data quality, reliability, and scalability across the platform.

Formación

• Proficient in designing and optimizing Spark SQL jobs for batch and real-time processing.
• Experience building ETL pipelines and data transformation frameworks.
• Skilled in performance tuning and optimization of Spark applications.
• Collaborates with business and technical teams to deliver data solutions.
• Ensures data quality, reliability, and scalability of big data platforms.

Responsabilidades

• Develop and maintain large-scale data processing applications using Scala and Apache Spark.
• Design and optimize Spark SQL jobs for batch and real-time data processing.
• Work with Hadoop ecosystem components such as HDFS, Hive, YARN, and MapReduce.
• Build ETL pipelines and data transformation frameworks.
• Perform performance tuning and optimization of Spark applications.
• Collaborate with business and technical teams to deliver data solutions.
• Ensure data quality, reliability, and scalability of Big Data platforms.

Conocimientos

Scala

Apache Spark

Spark SQL

ETL pipelines

Performance tuning

Big data

Herramientas

HDFS

Hive

YARN

MapReduce

Descripción del empleo

Develop and maintain large-scale data processing applications using Scala and Apache Spark.

• Design and optimize Spark SQL jobs for batch and real-time data processing.
• Work with Hadoop ecosystem components such as HDFS, Hive, YARN, and MapReduce.
• Build ETL pipelines and data transformation frameworks.
• Perform performance tuning and optimization of Spark applications.
• Collaborate with business and technical teams to deliver data solutions.
• Ensure data quality, reliability, and scalability of Big Data platforms