Big Data Scala Engineer - Hadoop & Spark

Envision Technology Solutions; Charlotte (NC); Sur place; USD 110 000 - 150 000; Plein temps

Résumé du poste

Envision Technology Solutions is seeking a Big Data Engineer in Charlotte to design and maintain large-scale Hadoop-based pipelines. You will build data ingestion and processing frameworks using Scala, Spark, and Hadoop, and automate deployments with Shell scripts.

The role also involves developing robust RESTful services with Java and Spring Boot, optimizing Spark jobs for batch and real-time workloads, and working with HDFS, Hive, Sqoop, Kafka, and Oozie within a collaborative, enterprise

Qualifications

• Design, develop and maintain large-scale Big Data solutions using Hadoop ecosystem technologies.
• Develop data ingestion, transformation and processing pipelines with Scala, Spark and Hadoop.
• Create and maintain Shell scripts for automation and deployment.
• Build scalable backend services with Java and Spring Boot.
• Develop and optimize Spark jobs for batch and real-time processing.
• Work with HDFS, Hive, Sqoop, Kafka and Oozie components.

Responsabilités

• Design, develop, and maintain large-scale Big Data solutions.
• Develop data ingestion, transformation, and processing frameworks.
• Create and maintain Shell scripts for automation and operations.
• Build backend services using Java and Spring Boot.
• Optimize Spark applications for batch and real-time processing.
• Collaborate with analysts, architects, DevOps, and stakeholders.

Connaissances

Hadoop

Scala

Shell scripting

Java

Spring Boot

Outils

Spark

Hive

Sqoop

Kafka

Oozie

HDFS

Description du poste

Envision Technology Solutions is seeking a Big Data Engineer in Charlotte to design and maintain large-scale Hadoop-based pipelines. You will build data ingestion and processing frameworks using Scala, Spark, and Hadoop, and automate deployments with Shell scripts.

The role also involves developing robust RESTful services with Java and Spring Boot, optimizing Spark jobs for batch and real-time workloads, and working with HDFS, Hive, Sqoop, Kafka, and Oozie within a collaborative, enterprise