Remote Big Data Engineer — Spark, Hadoop & Cloud Analytics

Bright Vision Technologies; Artesia, Cerritos (CA, CA); A distancia; USD 90.000 - 110.000; Jornada completa

Descripción de la vacante

Bright Vision Technologies is seeking an experienced Big Data Engineer to design, build, and operate large-scale data processing pipelines on Hadoop-based ecosystems. You will ingest, transform, and analyze massive volumes of structured and unstructured data to support analytics, ML workloads, and reporting.

The ideal candidate has 6+ years of hands-on experience with Spark, Hive, HDFS, and streaming platforms like Kafka or Spark Streaming, plus strong SQL, Python, and shell scripting skills.

Formación

• Bachelor’s degree in Computer Science, Engineering, or related technical discipline.
• 5+ years designing and operating big-data pipelines on Hadoop.
• Hands-on Apache Spark experience in production (Scala, Python, or Java).
• Solid Hive, HDFS, Sqoop, HBase, and Hadoop ecosystem experience.
• Hands-on streaming data platforms such as Kafka, Spark Streaming, or Flink.
• Strong SQL skills with relational and NoSQL data stores.
• Experience with Airflow or Oozie for workflow orchestration.
• Understanding of distributed systems: partitioning, replication, fault tolerance.
• Strong Python or Shell scripting abilities.
• Excellent troubleshooting, debugging, and documentation.

Responsabilidades

• Design, build, and operate large-scale data processing pipelines and analytics platforms on Hadoop ecosystems.
• Ingest, transform, and analyze massive volumes of structured and unstructured data to support analytics, ML, and reporting workloads.

Conocimientos

SQL

Troubleshooting

Documentation

Python scripting

Shell scripting

Distributed systems concepts

Educación

Bachelor’s degree in Computer Science, Engineering, or a related technical discipline

Herramientas

Apache Spark

Hive

HDFS

Sqoop

HBase

Kafka

Spark Streaming

Flink

Airflow

Oozie

Python

Shell

Descripción del empleo

Bright Vision Technologies is seeking an experienced Big Data Engineer to design, build, and operate large-scale data processing pipelines on Hadoop-based ecosystems. You will ingest, transform, and analyze massive volumes of structured and unstructured data to support analytics, ML workloads, and reporting.

The ideal candidate has 6+ years of hands-on experience with Spark, Hive, HDFS, and streaming platforms like Kafka or Spark Streaming, plus strong SQL, Python, and shell scripting skills.