Remote Hadoop Solutions Engineer

Bright Vision Technologies; Sugar Land (TX); Remote; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking an experienced Hadoop Solutions Developer to design, build, and operate large-scale data pipelines on Hadoop and related big-data ecosystems. You will ingest, transform, and analyze massive data to support analytics and reporting workloads with reliable performance.

You will work closely with data scientists and analysts, implement governance and quality controls, and mentor junior engineers while delivering scalable data platforms in production environments.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related technical discipline.
• Five or more years of professional experience designing and operating big-data pipelines on Hadoop.
• Strong hands-on expertise with Apache Spark (Scala, Python, or Java) in production environments.
• Solid experience with Hive, HDFS, Sqoop, HBase, and the broader Hadoop ecosystem.
• Hands-on experience with streaming data platforms such as Kafka, Spark Streaming, or Flink.

Responsibilities

• Design, develop, and operate end-to-end big-data pipelines on Hadoop.
• Build robust ETL/ELT workflows using Spark, Hive, Pig, and Sqoop.
• Develop high-throughput streaming data pipelines using Kafka, Spark Streaming, or Flink.
• Optimize Spark and MapReduce jobs to meet SLAs at minimal cost.
• Design and maintain data models and storage layouts on HDFS, Hive, HBase, and lakehouse formats.
• Implement data governance, lineage, and quality controls with governance teams.
• Build monitoring, alerting, and logging strategies for big-data pipelines.
• Mentor junior engineers and contribute to engineering standards.

Skills

Big-data pipelines on Hadoop

Spark (Scala/Python/Java)

Hive/HDFS/HBase

Streaming data platforms (Kafka/Spark 

SQL skills

Workflow orchestration (Airflow/Oozie)

Distributed systems concepts

Python/Shell scripting

Education

Bachelor’s degree in Computer Science, Engineering

Tools

Spark

Hive

HDFS

Sqoop

Kafka

Flink

Airflow

Oozie

Kubernetes

CI/CD / IaC

Job description

Bright Vision Technologies is seeking an experienced Hadoop Solutions Developer to design, build, and operate large-scale data pipelines on Hadoop and related big-data ecosystems. You will ingest, transform, and analyze massive data to support analytics and reporting workloads with reliable performance.

You will work closely with data scientists and analysts, implement governance and quality controls, and mentor junior engineers while delivering scalable data platforms in production environments.