Remote Big Data Engineer: Spark & Hadoop Expert

Bright Vision Technologies; Yuba City (CA); Remote; USD 90,000 - 110,000; Full time

Job summary

Bright Vision Technologies is seeking a highly skilled Big Data Engineer to design, build, and operate large-scale data processing pipelines on Hadoop and related ecosystems. You will ingest, transform, and analyze massive data sets to support analytics, ML, and reporting workloads in a fully remote environment across the U.S.

The ideal candidate has 6+ years of experience with Spark, Hive, HDFS, and streaming platforms, plus strong Python or Shell scripting and SQL.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related technical discipline.
• 5+ years designing and operating big-data pipelines on Hadoop.
• Strong hands-on expertise with Apache Spark (Scala, Python, or Java) in production environments.
• Solid experience with Hive, HDFS, Sqoop, HBase, and the broader Hadoop ecosystem.
• Hands-on experience with streaming data platforms such as Kafka, Spark Streaming, or Flink.
• Strong SQL skills and experience with relational and NoSQL data stores.
• Experience with workflow orchestration tools such as Airflow or Oozie.
• Understanding of distributed systems concepts such as partitioning, replication, and fault tolerance.
• Strong scripting in Python or Shell; excellent troubleshooting and documentation skills.

Responsibilities

• Design and operate large-scale data pipelines on Hadoop and related ecosystems.
• Ingest, transform, and analyze massive structured and unstructured data for analytics and ML workloads.
• Deliver reliable, performant, and cost-effective data platforms in production environments.

Skills

Big Data

Apache Spark

SQL

Python

Shell

Hadoop

Kafka/Streaming

Airflow/Oozie

Education

Bachelor’s degree in Computer Science or related field

6+ years of experience in big-data pipelines

Tools

Airflow

Oozie

Kafka

Hive

HDFS

Job description

Bright Vision Technologies is seeking a highly skilled Big Data Engineer to design, build, and operate large-scale data processing pipelines on Hadoop and related ecosystems. You will ingest, transform, and analyze massive data sets to support analytics, ML, and reporting workloads in a fully remote environment across the U.S.

The ideal candidate has 6+ years of experience with Spark, Hive, HDFS, and streaming platforms, plus strong Python or Shell scripting and SQL.