Senior Hadoop Data Engineer - Remote

Bright Vision Technologies; Wildwood (MO); Remote; USD 145,000 - 165,000; Full time

Job summary

Bright Vision Technologies is seeking a Senior Data Engineer - Hadoop to design, build, and operate large-scale data pipelines on Hadoop ecosystems for enterprise analytics. You will handle ingestion, transformation, and analysis of massive structured and unstructured data to support ML and reporting workloads.

The ideal candidate will have deep Hadoop ecosystem knowledge, strong software engineering fundamentals, and a track record delivering reliable, scalable data platforms in production

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related technical discipline.
• 6+ years of experience designing and operating big-data pipelines on Hadoop.
• Hands-on Apache Spark in production environments (Scala, Python, or Java).
• Experience with Hive, HDFS, Sqoop, HBase and the Hadoop ecosystem.
• Experience with streaming platforms: Kafka, Spark Streaming, or Flink.
• Strong SQL with Relational and NoSQL data stores.
• Airflow or Oozie for workflow orchestration.
• Strong Python or Shell scripting skills; good debugging and docs.

Responsibilities

• Design, build, and operate large-scale data pipelines.
• Ingest, transform, and analyze massive data volumes for analytics.
• Deliver reliable, performant, cost-effective data platforms in production.

Skills

Hadoop pipelines

Spark

Hadoop ecosystem

Kafka

Spark Streaming

SQL

Airflow

Oozie

Python

Shell scripting

Distributed systems

Documentation

Education

Bachelor’s degree in Computer Science or related field

Job description

Bright Vision Technologies is seeking a Senior Data Engineer - Hadoop to design, build, and operate large-scale data pipelines on Hadoop ecosystems for enterprise analytics. You will handle ingestion, transformation, and analysis of massive structured and unstructured data to support ML and reporting workloads.

The ideal candidate will have deep Hadoop ecosystem knowledge, strong software engineering fundamentals, and a track record delivering reliable, scalable data platforms in production