Senior Remote Big Data Engineer — Spark & Analytics

Socket.dev; Sunnyvale (CA); On-site; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking an experienced Big Data Engineer to design, build, and operate large-scale data processing pipelines on Hadoop and related ecosystems. This remote, full-time role supports enterprise analytics and ML workloads.

Ideal candidate has 6+ years of experience, strong Spark, Hive, and SQL skills, and hands-on experience with streaming platforms like Kafka and Spark Streaming. We offer a remote-friendly environment across the U.S.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related technical discipline.
• 5+ years of professional experience designing and operating big-data pipelines on Hadoop.
• Experience with Spark (Scala, Python, or Java) in production environments.
• Strong SQL skills and experience with relational and NoSQL data stores.
• Hands-on experience with streaming data platforms such as Kafka, Spark Streaming, or Flink.
• Experience with workflow orchestration tools such as Airflow or Oozie.

Responsibilities

• Design, build, and operate large-scale data processing pipelines on Hadoop ecosystems.
• Ingest, transform, and analyze massive volumes of data to support analytics, machine learning, and reporting workloads.
• Deliver reliable, performant, and cost-effective data platforms in production environments.

Skills

SQL proficiency

Troubleshooting skills

Documentation skills

Education

Bachelor’s degree in Computer Science, Engineering, or a related technical discipline

Tools

Apache Spark

Hive

HDFS

Sqoop

HBase

Kafka

Spark Streaming

Flink

Airflow

Oozie

Python

Shell

Job description

Bright Vision Technologies is seeking an experienced Big Data Engineer to design, build, and operate large-scale data processing pipelines on Hadoop and related ecosystems. This remote, full-time role supports enterprise analytics and ML workloads.

Ideal candidate has 6+ years of experience, strong Spark, Hive, and SQL skills, and hands-on experience with streaming platforms like Kafka and Spark Streaming. We offer a remote-friendly environment across the U.S.