Senior Big Data Engineer – HDFS, Spark & Hive

SmartRecruiters, Inc.; Durham (NC); On-site; USD 110,000 - 165,000; Full time

Job summary

Sonsoft, Inc. is a USA-based company seeking a senior data engineering expert with 7+ years of IT experience. The role focuses on HDFS architecture, YARN, ZooKeeper, and MapReduce, plus hands-on Hive, Sqoop, Spark, and data warehousing concepts. Experience with Oozie workflows and Cloudera distribution is a plus.

The candidate should be proficient in Java, Python, and Perl, and able to work in a fast-growing IT services environment in the United States.

Qualifications

• At least 7+ years of overall experience.
• HDFS architecture and understanding of nodes, YARN, ZooKeeper, and MapReduce.
• Hands-on Hive including queries, UDF, and file formats ORC, AVRO, Parquet.
• Hands-on Sqoop and Spark experience.

Skills

HDFS

YARN

ZooKeeper

MapReduce

Hive

Hive UDF

Parquet

ORC

Avro

Sqoop

Spark

Oozie

Cloudera

Flume

Kafka

Java

Python

Perl

Education

Bachelor's degree or foreign equivalent

4+ years IT experience

Tools

Oozie

Hadoop

Cloudera

Job description

Sonsoft, Inc. is a USA-based company seeking a senior data engineering expert with 7+ years of IT experience. The role focuses on HDFS architecture, YARN, ZooKeeper, and MapReduce, plus hands-on Hive, Sqoop, Spark, and data warehousing concepts. Experience with Oozie workflows and Cloudera distribution is a plus.

The candidate should be proficient in Java, Python, and Perl, and able to work in a fast-growing IT services environment in the United States.