Senior Hadoop Data Engineer | Remote Analytics

Bright Vision Technologies; Flower Mound (TX); On-site; USD 145,000 - 165,000; Full time

Job summary

Bright Vision Technologies is seeking a Senior Data Engineer - Hadoop to design, build, and operate large-scale data processing pipelines on Hadoop and related ecosystems for enterprise analytics and reporting.

You will transform massive structured and unstructured data, apply Spark, Hive, and streaming technologies, and collaborate with software engineering to deliver reliable, cost-effective data platforms in production.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related field.
• 5+ years designing and operating big-data pipelines on Hadoop.
• Strong hands-on Spark (Scala, Python, or Java).
• Experience with Hive, HDFS, Sqoop, HBase and Hadoop ecosystem.
• Hands-on with Kafka, Spark Streaming, or Flink.
• Strong SQL; experience with relational and NoSQL stores.
• Airflow or Oozie for workflows.
• Python or Shell scripting.
• Excellent debugging and documentation skills.

Skills

Spark

SQL

Distributed systems

Python

Shell

Troubleshooting

NoSQL

Data pipelines

Education

Bachelor's degree in CS/Engineering

Tools

Hive

HDFS

Sqoop

HBase

Kafka

Spark Streaming

Flink

Airflow

Oozie

AWS EMR

Azure HDInsight

Databricks

Delta Lake

Iceberg

Kubernetes

Job description

Bright Vision Technologies is seeking a Senior Data Engineer - Hadoop to design, build, and operate large-scale data processing pipelines on Hadoop and related ecosystems for enterprise analytics and reporting.

You will transform massive structured and unstructured data, apply Spark, Hive, and streaming technologies, and collaborate with software engineering to deliver reliable, cost-effective data platforms in production.