Remote Hadoop Big Data Engineer

United States Digital Space LLC; United States; Remote; USD 86,000 - 103,000; Full time

Job summary

Bright Vision Technologies is seeking an experienced Hadoop Big Data Developer for a 100% remote position based in the U.S. The role focuses on designing and operating large-scale data pipelines on Hadoop, enabling analytics, ML, and reporting workloads.

The ideal candidate has 6+ years of experience and strong skills in Spark, Hive/HDFS, SQL, and streaming platforms like Kafka or Flink. Prior experience with Airflow or Oozie and distributed systems concepts is essential.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related technical discipline.
• 6+ years of professional experience designing and operating big-data pipelines on Hadoop.
• Strong hands-on expertise with Apache Spark (Scala, Python, or Java) in production environments.
• Solid experience with Hive, HDFS, Sqoop, HBase, and the broader Hadoop ecosystem.
• Hands-on experience with streaming data platforms such as Kafka, Spark Streaming, or Flink.
• Strong SQL skills and experience working with both relational and NoSQL data stores.
• Experience with workflow orchestration tools such as Airflow or Oozie.
• Solid understanding of distributed systems concepts, including partitioning, replication, and fault tolerance.
• Strong scripting skills in Python or Shell.
• Excellent troubleshooting, debugging, and documentation skills.

Responsibilities

• Design, build, and operate large-scale data pipelines on Hadoop.
• Ingest, transform, and analyze massive volumes of data to support analytics and reporting workloads.
• Deliver reliable, performant, and cost-efficient data platforms in production environments.
• Support enterprise analytics, ML, and reporting use cases.

Skills

Spark

Hive/HDFS

SQL

Airflow

Oozie

Python

Shell

Kafka

Flink

Distributed systems

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

Airflow

Oozie

Kubernetes

Job description

Bright Vision Technologies is seeking an experienced Hadoop Big Data Developer for a 100% remote position based in the U.S. The role focuses on designing and operating large-scale data pipelines on Hadoop, enabling analytics, ML, and reporting workloads.

The ideal candidate has 6+ years of experience and strong skills in Spark, Hive/HDFS, SQL, and streaming platforms like Kafka or Flink. Prior experience with Airflow or Oozie and distributed systems concepts is essential.