Senior Data Engineer - Hadoop for Scalable Pipelines (Remote)

Bright Vision Technologies; Overland Park (KS); Remote; USD 145,000 - 165,000; Full time

Job summary

Bright Vision Technologies is seeking a Senior Data Engineer - Hadoop to design, build, and operate large-scale data processing pipelines on Hadoop and related ecosystems. You will ingest, transform, and analyze massive data to support analytics, machine learning, and reporting workloads.

The ideal candidate has deep Hadoop experience, strong software engineering fundamentals, and the ability to deliver reliable, cost-efficient data platforms in production environments.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related technical discipline.
• 5+ years designing and operating big-data pipelines on Hadoop.
• Strong hands-on Apache Spark (Scala, Python, or Java) in production.
• Experience with Hive, HDFS, Sqoop and the broader Hadoop ecosystem.
• Hands-on with streaming platforms such as Kafka, Spark Streaming, or Flink.
• Strong SQL skills with relational and NoSQL stores.
• Experience with Airflow or Oozie for workflow orchestration.
• Understanding of distributed systems concepts: partitioning, replication, fault tolerance.
• Scripting in Python or Shell; strong troubleshooting and documentation.

Responsibilities

• Design, build, and operate large-scale data processing pipelines on Hadoop ecosystems.
• Ingest, transform, and analyze massive data volumes for analytics and ML.
• Deliver reliable, performant, cost-effective data platforms in production.

Skills

Hadoop

Apache Spark

Hive

HDFS

Sqoop

HBase

Kafka

Flink

SQL

Airflow

Oozie

Python

Shell

Distributed systems

Troubleshooting

Documentation

Education

Bachelor’s degree in Computer Science, Engineering, or related technical discipline

Tools

Airflow

Oozie

Kubernetes

CI/CD

Infrastructure as Code

Job description

Bright Vision Technologies is seeking a Senior Data Engineer - Hadoop to design, build, and operate large-scale data processing pipelines on Hadoop and related ecosystems. You will ingest, transform, and analyze massive data to support analytics, machine learning, and reporting workloads.

The ideal candidate has deep Hadoop experience, strong software engineering fundamentals, and the ability to deliver reliable, cost-efficient data platforms in production environments.