Remote Data Platform Engineer - Big Data & Analytics

Bright Vision Technologies; Chapel Hill (NC); On-site; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies in the United States is seeking an experienced Data Platform Engineer to design, build, and operate large-scale data pipelines on Hadoop and related ecosystems. You will ingest, transform, and analyze massive data volumes to support analytics, machine learning, and reporting workloads.

The ideal candidate combines deep Hadoop expertise with strong software engineering fundamentals to deliver reliable, cost-efficient data platforms in production environments.

Qualifications

• 5+ years designing and operating big-data pipelines on Hadoop.
• Hands-on Spark experience in production (Scala, Python, or Java).
• Strong Hive, HDFS, Sqoop, and Hadoop ecosystem knowledge.
• Experience with streaming platforms like Kafka, Spark Streaming, or Flink.
• Strong SQL and experience with relational and NoSQL stores.
• Experience with Airflow or Oozie for workflow orchestration.
• Understanding of distributed systems including partitioning, replication, fault tolerance.
• Strong Python or Shell scripting skills.
• Excellent troubleshooting and documentation abilities.

Responsibilities

• Design, build, and operate large-scale data processing pipelines on Hadoop.
• Develop analytics platforms supporting enterprise analytics, ML, and reporting workloads.
• Deliver reliable, performant, and cost-effective data platforms in production.
• Collaborate with data engineers and data scientists to optimize data flows.

Skills

Big-data pipelines design/ops

Apache Spark (Scala/Python/Java)

SQL

Python scripting

Shell scripting

Distributed systems concepts

Tools

Hive

HDFS

Sqoop

HBase

Kafka

Spark Streaming

Flink

Airflow

Oozie

Job description

Bright Vision Technologies in the United States is seeking an experienced Data Platform Engineer to design, build, and operate large-scale data pipelines on Hadoop and related ecosystems. You will ingest, transform, and analyze massive data volumes to support analytics, machine learning, and reporting workloads.

The ideal candidate combines deep Hadoop expertise with strong software engineering fundamentals to deliver reliable, cost-efficient data platforms in production environments.