Senior Data Platform Engineer (Remote) - Hadoop & Spark

Bright Vision Technologies; Alpharetta (GA); Remote; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking a Data Platform Engineer to design, build, and operate large-scale data pipelines on Hadoop and related ecosystems while supporting analytics, ML, and reporting workloads. The role emphasizes reliability, performance, and cost efficiency in production environments, with a focus on cloud-enabled data platforms and modern data engineering practices.

The ideal candidate will bring deep expertise across the Hadoop ecosystem and strong software engineering

Qualifications

• 5+ years of professional experience with big-data pipelines on Hadoop.
• Hands-on Spark experience in production (Scala, Python, or Java).
• Solid experience with Hive, HDFS, Sqoop, and HBase in data platforms.
• Experience with streaming platforms such as Kafka, Spark Streaming, or Flink.
• Strong SQL and experience with relational and NoSQL stores.
• Experience with Airflow or Oozie for workflow orchestration.
• Solid understanding of distributed systems, including partitioning and fault tolerance.
• Strong Python or Shell scripting skills and strong debugging skills.

Responsibilities

• Design, build, and operate large-scale data processing pipelines and analytics platforms on Hadoop and related ecosystems.
• Collaborate with analytics, ML, and engineering teams to ensure reliable, performant, and cost-efficient data platforms.

Skills

Big-data pipelines design

Spark (Scala/Python/Java)

SQL & NoSQL data stores

Streaming data platforms

Workflow orchestration

Python or Shell scripting

Distributed systems fundamentals

Troubleshooting and documentation

Tools

Hadoop

Spark

Hive

HDFS

Sqoop

HBase

Kafka

Spark Streaming

Flink

Airflow

Oozie

Kubernetes

CI/CD

Infrastructure as Code

Job description

Bright Vision Technologies is seeking a Data Platform Engineer to design, build, and operate large-scale data pipelines on Hadoop and related ecosystems while supporting analytics, ML, and reporting workloads. The role emphasizes reliability, performance, and cost efficiency in production environments, with a focus on cloud-enabled data platforms and modern data engineering practices.

The ideal candidate will bring deep expertise across the Hadoop ecosystem and strong software engineering