Senior Remote Hadoop Solutions Engineer

Bright Vision Technologies; United States; Remote; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies seeks an experienced Hadoop Solutions Developer to design, build, and operate large‑scale data pipelines on Hadoop and related ecosystems. You will ingest, transform, and analyze massive structured and unstructured data to support analytics, ML, and reporting workloads.

The ideal candidate blends deep Hadoop expertise with strong software engineering for reliable, scalable, cost‑effective platforms.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related technical discipline.
• Five+ years designing and operating big‑data pipelines on Hadoop.
• Hands‑on Spark (Scala, Python, or Java) in production environments.
• Experience with Hive, HDFS, Sqoop, HBase and the Hadoop ecosystem.
• Experience with streaming data platforms such as Kafka, Spark Streaming, or Flink.
• Strong SQL skills with relational and NoSQL data stores.
• Workflow orchestration with Airflow or Oozie.
• Scripting in Python or Shell; excellent troubleshooting, debugging and documentation.

Responsibilities

• Design, develop, and operate end‑to‑end big‑data pipelines on Hadoop.
• Build ETL/ELT workflows with Spark, Hive, Pig, and Sqoop.
• Develop streaming pipelines using Kafka, Spark Streaming, or Flink.
• Tune Spark/MapReduce jobs for performance and cost efficiency.
• Design data models/storage on HDFS, Hive, HBase; support lakehouse formats.
• Implement data governance, lineage, and quality controls.
• Monitor, alert, and log big‑data pipelines with SLAs.
• Collaborate with data scientists to provide reliable datasets.
• Automate orchestration with Airflow, Oozie, or similar tools.
• Evaluate new technologies (EMR, Databricks, BigQuery) for improvements.
• Lead performance reviews and architecture audits of pipelines.
• Document architectures, schemas, and runbooks; mentor junior engineers.

Skills

Apache Spark

Scala

Python

Java

Hive

HDFS

Sqoop

HBase

Kafka

Spark Streaming

Flink

SQL

Airflow

Oozie

Python scripting

Shell scripting

Troubleshooting

Debugging

Documentation

Education

Bachelor's degree in Computer Science, Engineering, or a related technical discipline

Job description

Bright Vision Technologies seeks an experienced Hadoop Solutions Developer to design, build, and operate large‑scale data pipelines on Hadoop and related ecosystems. You will ingest, transform, and analyze massive structured and unstructured data to support analytics, ML, and reporting workloads.

The ideal candidate blends deep Hadoop expertise with strong software engineering for reliable, scalable, cost‑effective platforms.