Remote Data Platform Engineer | Hadoop & Spark Scale

Bright Vision Technologies; United States; Teletrabalho; USD 100 000 - 150 000; Tempo integral

Resumo da oferta

Bright Vision Technologies is seeking an experienced Data Platform Engineer to design, build, and operate large-scale data pipelines on Hadoop and related ecosystems. You will ingest, transform, and analyze massive volumes of structured and unstructured data to support analytics, machine learning, and reporting workloads.

The ideal candidate will bring deep Hadoop expertise, strong software engineering fundamentals, and a proven ability to deliver reliable, cost-efficient data platforms in

Qualificações

• 5+ years designing and operating big-data pipelines on Hadoop.
• Hands-on Apache Spark in production (Scala, Python, or Java).
• Experience with Hive, HDFS, Sqoop, HBase, and the Hadoop ecosystem.
• Experience with streaming platforms such as Kafka, Spark Streaming, or Flink.
• Strong SQL and experience with relational and NoSQL stores.
• Experience with Airflow or Oozie for workflow orchestration.
• Distributed systems concepts including partitioning, replication, and fault tolerance.
• Strong Python or Shell scripting skills.
• Excellent troubleshooting, debugging, and documentation skills.

Conhecimentos

Big-data pipelines

Spark

Hive/HDFS

Streaming (Kafka/Spark Streaming/Flink

SQL

Airflow/Oozie

Distributed systems

Python/Shell

Troubleshooting

Descrição da oferta de emprego

Bright Vision Technologies is seeking an experienced Data Platform Engineer to design, build, and operate large-scale data pipelines on Hadoop and related ecosystems. You will ingest, transform, and analyze massive volumes of structured and unstructured data to support analytics, machine learning, and reporting workloads.

The ideal candidate will bring deep Hadoop expertise, strong software engineering fundamentals, and a proven ability to deliver reliable, cost-efficient data platforms in