Remote Big Data Engineer & Pipeline Expert

Bright Vision Technologies; United States; Remote; USD 100,000 - 150,000; Full time

Benefits offered by this job

Remote work

Job summary

Bright Vision Technologies, a technology consulting and software company, seeks a Big Data Application Developer for remote work in the United States. You will design and operate large-scale data pipelines on Hadoop ecosystems, enabling analytics, ML, and reporting workloads.

Ideal candidates have 6+ years in big-data design and strong Spark, SQL, and Python skills, with experience in streaming, workflows, and distributed systems for scalable, production-grade platforms.

Qualifications

• Bachelor’s degree in CS, engineering, or related field.
• 5+ years designing and operating big-data pipelines on Hadoop.
• Hands-on Spark (Scala/Python/Java) in production.
• Experience with Hive, HDFS, Sqoop, HBase.
• Experience with streaming platforms (Kafka, Spark Streaming, Flink).
• Strong SQL for relational and NoSQL stores.
• Experience with Airflow or Oozie for workflows.
• Solid understanding of distributed systems concepts.
• Scripting in Python or Shell; good troubleshooting and documentation.

Responsibilities

• Design, build, and operate large-scale data pipelines on Hadoop.
• Develop analytics platforms to support enterprise analytics and ML workloads.
• Ensure reliable, performant, cost-effective data platforms in production.

Skills

Big-data pipeline design

Apache Spark

SQL

Python scripting

Shell scripting

Troubleshooting

Documentation

Distributed systems concepts

Education

Bachelor's degree in Computer Science, Engineering, or a related technical discipline

Tools

Hive

HDFS

Sqoop

HBase

Kafka

Spark Streaming

Flink

Airflow

Oozie

Job description

Bright Vision Technologies, a technology consulting and software company, seeks a Big Data Application Developer for remote work in the United States. You will design and operate large-scale data pipelines on Hadoop ecosystems, enabling analytics, ML, and reporting workloads.

Ideal candidates have 6+ years in big-data design and strong Spark, SQL, and Python skills, with experience in streaming, workflows, and distributed systems for scalable, production-grade platforms.