Remote Hadoop Big Data Engineer: Spark & Pipelines

Bright Vision Technologies

Naperville (IL)

Remote

USD 86.000 - 103.000

Vollzeit

Vor 4 Tagen

Sei unter den ersten Bewerbenden

Bewerbungsgenerator

Hebe dich für diese Rolle von der Masse ab — erstelle in etwa einer Minute einen maßgeschneiderten Lebenslauf und ein Anschreiben.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

Bright Vision Technologies is seeking an experienced Hadoop Big Data Developer to design, build, and operate large-scale data processing pipelines on Hadoop and related ecosystems. This is a 100% remote position for U.S.-based candidates.

In this role you will ingest, transform, and analyze massive volumes of structured and unstructured data to support analytics, machine learning, and reporting workloads. The ideal candidate has 6+ years of experience with Spark, Hive, HDFS, and related tools,

Qualifikationen

Bachelor's degree required in CS/Engineering or related field.
5+ years of experience designing and operating big-data pipelines on Hadoop.
Strong hands-on expertise with Apache Spark (Scala, Python, or Java) in production.
Solid experience with Hive, HDFS, Sqoop, HBase, and the broader Hadoop ecosystem.
Hands-on experience with streaming data platforms such as Kafka, Spark Streaming, or Flink.
Strong SQL skills and experience with relational and NoSQL data stores.
Experience with workflow orchestration tools such as Airflow or Oozie.
Solid understanding of distributed systems concepts including partitioning, replication, and fault tolerance.
Strong scripting skills in Python or Shell.
Excellent troubleshooting, debugging, and documentation skills.

Kenntnisse

Spark (Scala/Python/Java)

Hive

HDFS

Sqoop

HBase

Kafka

Spark Streaming

Flink

SQL

Airflow

Oozie

Python scripting

Shell scripting

Distributed systems

Troubleshooting

Ausbildung

Bachelor's degree in Computer Science, Engineering, or related field

Tools

Apache Spark