Senior Hadoop Big Data Engineer—Remote

Bright Vision Technologies; Farmington (CT); Remote; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking an experienced Hadoop Big Data Developer to design, build, and operate large-scale data pipelines on Hadoop and related ecosystems. This role focuses on ingesting, transforming, and analyzing massive data to support analytics, ML, and reporting workloads.

The ideal candidate combines deep Hadoop ecosystem expertise with strong software engineering practices, delivering reliable, scalable, and cost-efficient data platforms in production.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related technical discipline.
• 10+ years of professional experience designing and operating big-data pipelines on Hadoop.
• Strong hands-on expertise with Apache Spark (Scala, Python, or Java) in production environments.
• Solid experience with Hive, HDFS, Sqoop, HBase, and the broader Hadoop ecosystem.
• Hands-on experience with streaming data platforms such as Kafka, Spark Streaming, or Flink.
• Strong SQL skills and experience working with both relational and NoSQL data stores.
• Experience with workflow orchestration tools such as Airflow or Oozie.
• Solid understanding of distributed systems concepts, including partitioning, replication, and fault tolerance.
• Strong scripting skills in Python or Shell.
• Excellent troubleshooting, debugging, and documentation skills.

Responsibilities

• Design, build, and operate large-scale data pipelines on Hadoop.
• Ingest, transform, and analyze massive volumes of data to support analytics workloads.
• Deliver reliable, performant, and cost-effective data platforms in production environments.

Skills

Spark

Hadoop

Hive

HDFS

Sqoop

HBase

Kafka

Spark Streaming

Flink

SQL

NoSQL

Airflow

Oozie

Python

Shell

Troubleshooting

Documentation

Education

Bachelor’s degree in Computer Science, Engineering, or related technical discipline

Tools

Airflow

Oozie

Kubernetes

Spark-on-K8s

Job description

Bright Vision Technologies is seeking an experienced Hadoop Big Data Developer to design, build, and operate large-scale data pipelines on Hadoop and related ecosystems. This role focuses on ingesting, transforming, and analyzing massive data to support analytics, ML, and reporting workloads.

The ideal candidate combines deep Hadoop ecosystem expertise with strong software engineering practices, delivering reliable, scalable, and cost-efficient data platforms in production.