Remote Big Data Engineer - Scale Pipelines & Analytics

United States Digital Space LLC; United States; Remote; USD 100,000 - 150,000; Full time

Benefits offered by this job

Remote work

Job summary

Bright Vision Technologies is seeking an experienced Big Data Engineer for a 100% remote role in the United States. The candidate will design, build, and operate large-scale data pipelines on Hadoop ecosystems, ingesting data from diverse sources and delivering reliable analytics platforms.

This role demands strong software engineering and cost‑aware production discipline. The ideal candidate has 6+ years in big data, proficiency with Spark, Hive, HDFS, and streaming tools, plus scripting skills

Qualifications

• 5+ years of professional experience designing and operating big‑data pipelines on Hadoop.
• Strong hands‑on expertise with Apache Spark in production environments (Scala, Python, or Java).
• Solid experience with Hive, HDFS, Sqoop, and the broader Hadoop ecosystem.
• Hands‑on experience with streaming data platforms such as Kafka, Spark Streaming, or Flink.

Responsibilities

• Design, develop, and operate end‑to‑end big‑data pipelines on Hadoop.
• Build robust ETL/ELT workflows with Spark, Hive, Sqoop.
• Develop high‑throughput streaming data pipelines using Kafka or Flink.
• Optimize Spark/MapReduce jobs for performance and cost.
• Design and maintain data models on HDFS, Hive, and lakehouse formats.
• Implement data governance and quality controls with governance teams.
• Build monitoring, alerting, and logging for pipelines.
• Collaborate with data scientists to provide curated datasets.
• Automate orchestration using Airflow or Oozie.

Skills

SQL proficiency

Distributed systems understanding

Programming in Python/Scala/Java

Troubleshooting & debugging

Documentation skills

Education

Bachelor's degree in Computer Science or related field

Tools

Hadoop

Apache Spark

Hive

HDFS

Sqoop

Kafka

Spark Streaming

Flink

Airflow

Oozie

Databricks

Delta Lake / Iceberg / Hudi

Job description

Bright Vision Technologies is seeking an experienced Big Data Engineer for a 100% remote role in the United States. The candidate will design, build, and operate large-scale data pipelines on Hadoop ecosystems, ingesting data from diverse sources and delivering reliable analytics platforms.

This role demands strong software engineering and cost‑aware production discipline. The ideal candidate has 6+ years in big data, proficiency with Spark, Hive, HDFS, and streaming tools, plus scripting skills