Remote Hadoop Data Pipeline Engineer

United States Digital Space LLC; United States; Remote; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking an experienced Hadoop Solutions Developer to design, build, and operate large-scale data pipelines on Hadoop and related ecosystems. This remote, full-time role focuses on ingesting, transforming, and analyzing massive data for enterprise analytics and reporting workloads.

The ideal candidate brings deep Hadoop experience, strong software engineering fundamentals, and a track record of reliable, cost-efficient production platforms.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related technical discipline.
• 5+ years of professional experience designing and operating big-data pipelines on Hadoop.
• Strong hands-on expertise with Apache Spark (Scala, Python, or Java) in production environments.
• Solid experience with Hive, HDFS, Sqoop, HBase, and the broader Hadoop ecosystem.
• Hands-on experience with streaming data platforms such as Kafka, Spark Streaming, or Flink.

Responsibilities

• Design, develop, and operate end-to-end big-data pipelines on Hadoop, ingesting data from diverse sources.
• Build robust ETL/ELT workflows using Spark, Hive, Pig, and Sqoop with data quality and recoverability.
• Develop high-throughput streaming data pipelines using Kafka, Spark Streaming, or Flink.
• Optimize Spark and MapReduce jobs for performance and cost efficiency.
• Design data models and storage layouts on HDFS, Hive, HBase, and lakehouse formats.
• Implement data governance, lineage, and quality controls.

Education

Bachelor's degree in Computer Science or related

Tools

Spark

Hive

HDFS

Sqoop

Kafka

Airflow

Oozie

Delta Lake

Job description

Bright Vision Technologies is seeking an experienced Hadoop Solutions Developer to design, build, and operate large-scale data pipelines on Hadoop and related ecosystems. This remote, full-time role focuses on ingesting, transforming, and analyzing massive data for enterprise analytics and reporting workloads.

The ideal candidate brings deep Hadoop experience, strong software engineering fundamentals, and a track record of reliable, cost-efficient production platforms.