Remote Big Data Engineer: Spark & Hadoop

Bright Vision Technologies; Tampa (FL); Remote; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking an experienced Big Data Application Developer to design, build, and operate large-scale data processing pipelines and analytics platforms on Hadoop and related big-data ecosystems.

In this role you will be responsible for ingesting, transforming, and analyzing massive volumes of structured and unstructured data to support enterprise analytics, machine learning, and reporting workloads.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related technical discipline.
• 5+ years designing and operating big-data pipelines on Hadoop.
• Hands-on Apache Spark in production environments.
• Experience with Hive, HDFS, Sqoop, HBase in the Hadoop ecosystem.
• Experience with streaming platforms such as Kafka, Spark Streaming, or Flink.
• Strong SQL skills with relational and NoSQL stores.
• Experience with Airflow or Oozie for workflows.
• Understanding of distributed systems, partitioning, replication, fault tolerance.
• Scripting in Python or Shell.
• Excellent troubleshooting, debugging, and documentation skills.

Responsibilities

• Design, build, and operate large-scale data processing pipelines.
• Develop analytics platforms supporting enterprise analytics, ML, and reporting.
• Ingest, transform, and analyze massive volumes of structured and unstructured data.

Skills

Spark

Python

SQL

Shell

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

Hive

HDFS

Sqoop

HBase

Kafka

Spark Streaming

Flink

Airflow

Oozie

Job description

Bright Vision Technologies is seeking an experienced Big Data Application Developer to design, build, and operate large-scale data processing pipelines and analytics platforms on Hadoop and related big-data ecosystems.

In this role you will be responsible for ingesting, transforming, and analyzing massive volumes of structured and unstructured data to support enterprise analytics, machine learning, and reporting workloads.