Hadoop Data Engineer

Cloudious LLC; Dallas (TX); On-site; USD 120,000 - 180,000; Full time

Job summary

Cloudious LLC in Dallas, TX is seeking an experienced Hadoop Data Engineer with a strong background in Java, Spring Boot, Spark, and Scala to design, build, and optimize scalable data processing solutions for batch and real-time pipelines.

The ideal candidate will work across Hadoop ecosystem components, implement RESTful data services, tune Hive queries, and collaborate with cross-functional teams to translate business requirements into robust data platforms.

Qualifications

• 10+ years hands-on Java development experience.
• Expertise in Spring Boot 2 framework.
• Proficient in Spark and Scala for big data processing.
• Strong Hive SQL optimization and Hadoop MapReduce knowledge.
• Experience with Kafka messaging and streaming.
• RESTful web services design and consumption.
• Version control with GitLab.
• Build tools Maven/Gradle; CI/CD experience.
• Understanding distributed systems and big data architectures.
• Experience with JD Edwards EnterpriseOne is a plus.

Responsibilities

• Design, develop, and maintain scalable data processing apps using Java and Spring Boot.
• Build and optimize big data pipelines with Spark and Scala for large-scale processing.
• Develop and consume RESTful web services for data integration.
• Implement batch and streaming pipelines with Spark Streaming and Kafka.
• Write optimized Hive queries and SQL for performance.
• Work on distributed data platforms leveraging Hadoop components.
• Collaborate with cross-functional teams to translate requirements into solutions.
• Ensure code quality via reviews, version control, and CI/CD practices.
• Troubleshoot performance, scalability, and data reliability issues.

Skills

Java 8+

Spring Boot 2

Spark

Scala

Kafka

RESTful services

GitLab

Maven/Gradle

CI/CD

Distributed systems

Big data architecture

Tools

GitLab

Maven/Gradle

Hive

SQL

Spark Streaming

Job description

Job Role: Hadoop Data Engineer

Location: Dallas, TX

Duration: 6+ Months

ABOUT POSITION

Experience Required: 10+ years of hands-on experience in Java (version 8 or above).

We are seeking an experienced Data Engineer with a strong background in Java, Spring Boot, Spark, and Scala, along with a deep understanding of big data and distributed systems.

The ideal candidate will be responsible for designing, building, and optimizing scalable data processing solutions, supporting both batch and real-time data pipelines.

WHAT YOU'LL DO

• Design, develop, and maintain scalable data processing applications using Java (8 or above) and Spring Boot.
• Build and optimize big data pipelines using Spark and Scala for large-scale data processing.
• Develop and consume RESTful web services for data integration and platform interoperability.
• Implement and manage batch and streaming data pipelines using Spark Streaming and Kafka.
• Write optimized Hive queries and SQL, focusing on performance and scalability.
• Work on distributed data platforms leveraging Hadoop ecosystem components.
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
• Ensure code quality through reviews, version control, and CI/CD best practices.
• Troubleshoot and resolve performance, scalability, and data reliability issues

MANDATORY SKILLS

• 8 12 years of hands-on experience in Java (version 8 or above).
• Strong expertise in Spring Boot 2 framework.
• Excellent proficiency in Spark.
• Strong experience with Hive, SQL optimization, Hadoop Map Reduce, and Kafka Messaging Bus.
• Solid understanding of REST Web Services.
• Experience with version control systems such as Version Management GitLab.
• Hands-on experience with build tools like Maven and/or Gradle.
• Knowledge of CI CD & DevOps is a plus
• Strong understanding of distributed systems and big data architecture.
• Experience with streaming frameworks such as Spark Streaming and Kafka Messaging Bus.
• Experience working in large-scale JD Edwards EnterpriseOne.
• Exposure to Performance Tuning and SAP PMO Capacity planning for big data systems